21 hours ago
Base Salary
$144k - $194k/yr
Responsibilities
- Contribute core serving features to open-source inference frameworks such as vLLM and SGLang, including continuous batching, paged attention, quantization, and distributed inference on Neuron.
- Expand the range of models supported out of the box and build tooling and automation for production-ready deployment.
- Improve model enablement, testing, benchmarking, release, and shipping workflows.
- Collaborate with model development, performance, compiler, and runtime engineers to deliver accurate, scalable, and efficient inference.
- Apply code reviews, testing, and operational engineering practices to ship reliable high-performance software.
Requirements
- At least 3 years of non-internship professional software development experience.
- At least 3 years of non-internship experience designing or architecting new and existing systems, including design patterns, reliability, and scaling.
- At least 2 years of experience designing and developing large-scale, multi-tiered, multithreaded, embedded, or distributed applications, tools, systems, or services using C#, C++, Java, or Perl.
- Bachelor’s degree or foreign equivalent in Computer Science, Engineering, Mathematics, or a related field.
- Knowledge of machine learning and LLM fundamentals, including transformer architecture, training and inference lifecycles, and optimization techniques.
- Preferred experience includes debugging, profiling, large-scale systems best practices, production LLM deployment on GPUs, Neuron, TPU, or other AI acceleration hardware, and vLLM, SGLang, TensorRT, CUDA, or Triton.
- Preferred experience includes kernel development.
Benefits
- Comprehensive health insurance including medical, dental, vision, prescription, life, and AD&D coverage.
- Employee assistance, mental health support, medical advice line, flexible spending accounts, and adoption and surrogacy reimbursement.
- 401(k) matching, paid time off, and parental leave.
- Sign-on payments and restricted stock units may be included in the Amazon package.
- Work location is Seattle, Washington, USA.
Categories
About Amazon
Amazon builds and operates a global e-commerce marketplace, logistics network, and consumer devices, and provides cloud computing via AWS for businesses and developers. The company earns revenue from online retail, third‑party seller services, subscriptions like Prime, advertising, and AWS usage. Founded in 1994 and headquartered in Seattle, it is publicly traded on NASDAQ (AMZN) and serves customers in dozens of countries.