Senior ML Infrastructure Engineer
SambaNova Systems4 hours ago
Base Salary
$200k - $275k/yr
Responsibilities
- Design and productionize advanced inference techniques on RDU for performance and cost optimization.
- Develop capabilities involving speculative decoding, constrained decoding, function/tool calling, prompt caching, and long-context inference.
- Own SambaNova’s integration with vLLM and adjacent serving frameworks, adapting them to RDU architecture.
- Own the public inference API surface exposed through SambaStack and SambaCloud.
- Build and maintain accuracy verification and regression infrastructure for inference features.
- Partner with ML, compiler, runtime, and product teams to take features from prototype to production.
- Contribute to technical designs, code reviews, and architectural decisions as a senior individual contributor.
Requirements
- Bachelor’s degree in Computer Science, Electrical Engineering, or a related field.
- 5+ years of industry experience building and operating large-scale distributed systems, ideally in ML serving.
- Strong fundamentals in algorithms, data structures, concurrency, and systems design.
- Experience designing and maintaining production services with strict latency, throughput, and availability requirements.
- Working knowledge of modern LLM inference techniques.
- Familiarity with open-source serving stacks such as vLLM, TensorRT-LLM, or SGLang.
- Proficiency in Python.
- Experience collaborating across teams to deliver complex, system-level engineering solutions.
Benefits
- Base salary range of $200,000–$275,000 USD, plus equity for US-based full-time employment positions.
- SambaNova covers 95% of employee medical insurance premiums and 77% of dependent premiums, with a Health Savings Account and employer contribution.
- Dental, vision, short- and long-term disability, basic and voluntary life, AD&D, and flexible spending account options are offered.
- Well-being benefits include Headspace, Gympass+ access, One Medical membership, counseling services, and an Employee Assistance Program.
Tech Stack
Categories
About SambaNova Systems
SambaNova Systems builds full-stack AI infrastructure—custom accelerator chips, systems, and foundation models—delivered as on-prem appliances and cloud APIs for enterprises, governments, and data centers. Its offerings include the SambaNova Suite, Samba Cloud, and on-prem SambaStack, powered by its Reconfigurable Dataflow Unit (RDU) architecture, including the SN40L chip. Founded in 2017 and headquartered in San Jose, California, the company is privately held.