1 day ago
Redmond, WA, USAMid Level / Senior
H1B sponsor
Base Salary
$102k - $261k/yr
Responsibilities
- Own an AI inference serving component from design through production.
- Profile execution, memory, and dependency bottlenecks and improve throughput, latency, and cost.
- Improve cache-aware routing, admission control, continuous batching, and KV-cache management.
- Validate time-to-first-token, tail latency, and cost per token through load and failure testing.
- Build secure, tenant-isolated systems with automated testing, observability, recovery, and safe releases.
- Provide operational support and improve services using production feedback.
- Preserve model quality and numerical correctness through reproducible benchmarks.
- Pursue near-linear throughput scaling across supported GPU and node configurations.
- Build provider-aware serving stacks and integrate with training and post-training stacks for supported models.
Requirements
- Bachelor's degree in computer science or a related technical field and at least 2 years of technical engineering experience with coding, or equivalent experience.
- Preferred: at least 5 years of technical engineering experience with coding, or equivalent experience.
- Proven coding experience in C++, C#, Rust, Go, Java, or Python, with networking and concurrency fundamentals.
- Experience with software engineering fundamentals, debugging, and cross-team communication.
- Experience shipping and operating gateways, model-serving runtimes, or accelerator software.
- Experience with containerized services, Kubernetes, and serving runtimes such as vLLM, SGLang, or TensorRT-LLM.
- Ability to meet Microsoft, customer, and/or government security screening requirements, including the Microsoft Cloud background check.
Tech Stack
Categories
About Microsoft
Microsoft develops operating systems, productivity software, cloud services, developer tools, and consumer devices for individuals, enterprises, and governments. Its main products include Windows, Microsoft 365, Azure, Visual Studio/GitHub, Xbox, and LinkedIn; revenue comes from software subscriptions and licenses, cloud consumption, hardware sales, and advertising. Founded in 1975 and headquartered in Redmond, Washington, Microsoft is a public company traded on Nasdaq.
