1 day ago
Base Salary
$143k - $304k/yr
Responsibilities
- Lead software prototyping and proof-of-concept development for memory tiering, pooling, overcommit, and hardware/software co-designed capabilities.
- Characterize and optimize LLM inference workloads, including KV Cache capacity, placement, migration, utilization, and offload strategies.
- Design workload studies for agentic, multi-turn, coding, reasoning, and long-context AI workloads, measuring memory consumption, latency, throughput, token efficiency, and utilization.
- Analyze data movement across GPU, CPU, storage, and networking subsystems, including GPU Direct Storage, GPU Direct RDMA, peer-to-peer transfers, and distributed inference pipelines.
- Develop inference framework extensions, instrumentation, performance models, and simulation frameworks for large-scale AI deployments.
- Collaborate with workload experts, architects, hardware engineers, software leads, partner ISVs, and other stakeholders to shape Azure systems architecture and technical direction.
Requirements
- Bachelor’s degree in Computer Science or a related technical field and 6+ years of technical engineering experience involving coding, or equivalent experience.
- Preferred experience includes 12+ years in systems software and 10+ years leading significant hardware/software co-design projects.
- Deep expertise in Linux kernel internals, memory management, I/O subsystems, NUMA, DMA, and GPU, CPU, storage, and network data paths.
- Hands-on experience with NVIDIA GPU software stacks, including CUDA, NCCL, GPUDirect Storage, and GPUDirect RDMA.
- Experience with AI inference infrastructure, large GPU clusters, inference serving architectures, KV Cache-intensive workloads, inference runtimes, scheduling systems, and distributed inference.
- Experience with vLLM, SGLang, TensorRT-LLM, LMCache, SGLang HiCache, cache offload, cache sharing, memory tiering, CXL memory expansion, and memory pooling.
- Strong software development skills in C/C++, Python, and distributed systems, plus the ability to lead complex cross-disciplinary projects and influence technical direction.
- Ability to meet Microsoft Cloud Background Check and applicable customer or government security screening requirements.
- Strong collaboration, communication, analytical, problem-solving, stakeholder engagement, and ambiguity-management skills.
Categories
About Microsoft
Microsoft develops operating systems, productivity software, cloud services, developer tools, and consumer devices for individuals, enterprises, and governments. Its main products include Windows, Microsoft 365, Azure, Visual Studio/GitHub, Xbox, and LinkedIn; revenue comes from software subscriptions and licenses, cloud consumption, hardware sales, and advertising. Founded in 1975 and headquartered in Redmond, Washington, Microsoft is a public company traded on Nasdaq.