ML Infrastructure Engineer - New Grad
SambaNova Systems2 days ago
Base Salary
$131k - $197k/yr
Responsibilities
- Implement inference features on RDU accelerators, including speculative decoding, constrained decoding, tool calling, prompt caching, and long-context inference.
- Adapt vLLM and related serving frameworks to the RDU architecture.
- Build and maintain endpoints in the public inference API.
- Write accuracy verification and regression tests for inference features.
- Debug latency, throughput, and reliability issues in production services.
Requirements
- Bachelor’s degree in Computer Science, Electrical Engineering, or a related field.
- Strong Python programming skills and understanding of algorithms, data structures, concurrency, and systems design.
- Coursework or project experience in distributed systems or machine learning.
- Experience with LLM serving frameworks such as vLLM, SGLang, or TensorRT-LLM.
- Experience with inference techniques such as speculative decoding, KV caching, or continuous batching.
- Experience with PyTorch.
- Preferred qualifications include an M.S. or Ph.D., API or backend-service experience, ML systems or backend infrastructure internship or research experience, disaggregated prefill/decode systems experience, and open-source project ownership or contributions.
Benefits
- Base salary plus equity and benefits for US-based full-time employment.
- SambaNova covers 95% of employee medical insurance premiums and 77% of dependent premiums, with an employer-contributed Health Savings Account.
- Dental, vision, short- and long-term disability, life, AD&D, and flexible spending account options are offered.
- Well-being benefits include Headspace, Gympass+, One Medical, counseling services, and an Employee Assistance Program.
Categories
About SambaNova Systems
SambaNova Systems builds full-stack AI infrastructure—custom accelerator chips, systems, and foundation models—delivered as on-prem appliances and cloud APIs for enterprises, governments, and data centers. Its offerings include the SambaNova Suite, Samba Cloud, and on-prem SambaStack, powered by its Reconfigurable Dataflow Unit (RDU) architecture, including the SN40L chip. Founded in 2017 and headquartered in San Jose, California, the company is privately held.