1 day ago
Base Salary
$143k - $331k/yr
Responsibilities
- Own an inference service and lead delivery across gateway and runtime boundaries.
- Evaluate prefix caching, prefill/decode disaggregation, speculative decoding, quantization, and tensor and expert parallelism.
- Preserve streaming, cancellation, retries, token-accounting correctness, model quality and numerical correctness.
- Lead capacity planning, incident prevention, model onboarding and operational support while mentoring engineers.
- Build secure, tenant-isolated systems with automated testing, observability, reliable recovery and safe releases.
- Use reproducible benchmarks and production feedback to improve throughput, latency, capacity and cost.
- Build provider-aware serving stacks and integrate with training and post-training stacks for OpenAI, Anthropic, Microsoft AI and open-source models.
Requirements
- Bachelor's degree in computer science or a related technical field and at least 6 years of technical engineering experience with coding in C, C++, C#, Java, JavaScript or Python, or equivalent experience.
- Preferred: 8 or more years of technical engineering experience with coding in C, C++, C#, Java, JavaScript or Python, or equivalent experience.
- Preferred: proven distributed-services and model-serving delivery spanning at least two connected layers.
- Preferred: strong software engineering fundamentals, debugging skills and cross-team communication.
- Preferred: experience translating customer requirements into service objectives and measurable performance improvements.
- Preferred: experience with containerized services, Kubernetes and serving runtimes such as vLLM, SGLang or TensorRT-LLM.
- Ability to meet Microsoft, customer and/or government security screening requirements, including the Microsoft Cloud background check.
Tech Stack
Categories
About Microsoft
Microsoft develops operating systems, productivity software, cloud services, developer tools, and consumer devices for individuals, enterprises, and governments. Its main products include Windows, Microsoft 365, Azure, Visual Studio/GitHub, Xbox, and LinkedIn; revenue comes from software subscriptions and licenses, cloud consumption, hardware sales, and advertising. Founded in 1975 and headquartered in Redmond, Washington, Microsoft is a public company traded on Nasdaq.
