10 hours ago
Base Salary
$168k - $227k/yr
Responsibilities
- Lead customization and optimization of vLLM and SGLang on AWS Neuron.
- Develop core inference framework logic including scheduling and model execution.
- Apply kernel development, parallel computation, distributed KV cache, speculative decoding, and systems engineering techniques to improve LLM serving performance.
- Collaborate with model development, compiler, runtime, and performance engineering teams.
- Evaluate emerging inference research and translate it into production-ready features.
- Provide technical roadmap influence, design leadership, mentorship, and engineering best practices.
Requirements
- 5+ years of non-internship professional software development experience.
- 5+ years of programming experience with at least one software programming language.
- 5+ years leading design or architecture of new and existing systems, including reliability and scaling.
- 5+ years of full software development lifecycle experience, including coding standards, code reviews, source control management, build processes, testing, and operations.
- Experience mentoring, serving as a tech lead, or leading an engineering team.
- Fundamentals of machine learning models, architectures, training and inference lifecycles, plus experience optimizing model performance.
- Preferred: bachelor's degree in computer science or equivalent.
- Preferred: hands-on PyTorch or JAX experience, preferably developing and deploying LLMs in production on GPUs, AWS Neuron, TPUs, or other AI accelerator hardware.
- Preferred: experience extending or contributing to inference frameworks such as vLLM or SGLang.
- Preferred: kernel development experience with technologies such as CUDA or Triton.
Benefits
- Comprehensive health insurance including medical, dental, vision, prescription, Basic Life and AD&D insurance, and optional supplemental life plans.
- Employee assistance program, mental health support, medical advice line, flexible spending accounts, and adoption and surrogacy reimbursement coverage.
- 401(k) matching, paid time off, and parental leave.
- Sign-on payments and restricted stock units may be included in the Amazon package.
- Work location: Seattle, Washington, USA.
Tech Stack
Categories
About Amazon
Amazon builds and operates a global e-commerce marketplace, logistics network, and consumer devices, and provides cloud computing via AWS for businesses and developers. The company earns revenue from online retail, third‑party seller services, subscriptions like Prime, advertising, and AWS usage. Founded in 1994 and headquartered in Seattle, it is publicly traded on NASDAQ (AMZN) and serves customers in dozens of countries.