Software Engineer, LLM Infrastructure
Fireworks AI9 months ago
San Mateo, CA, USAMid Level
H1B Sponsor
Base Salary
$175k - $220k/yr
Responsibilities
- Design and develop scalable backend infrastructure supporting distributed training, inference, and data pipelines.
- Build and maintain backend services including the LLM pipeline, control plane, and model-serving systems.
- Improve performance, cost efficiency, and reliability across compute, storage, and networking layers.
- Build frameworks and safeguards to improve model quality.
- Collaborate with performance, training, product, cloud infrastructure, and related teams to translate research and product needs into infrastructure solutions.
- Participate in code reviews, technical discussions, and continuous integration and deployment processes.
Requirements
- Bachelor’s degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience.
- At least 3 years of software engineering experience focused on infrastructure or machine-learning systems.
- Strong programming skills in Python, Go, or a similar language.
- Experience with ML infrastructure and tooling such as PyTorch, MLflow, Vertex AI, SageMaker, or Kubernetes.
- Basic understanding of LLM concepts including context length, disaggregated prefill, and KV cache memory estimation.
- Preferred: 5+ years of software engineering experience focused on infrastructure or machine-learning systems.
- Preferred: experience with open-source inference engines such as vLLM, Sglang, or TRT-LLM.
- Preferred: contributions to open-source infrastructure or machine-learning projects.
- Preferred: experience building large-scale ML/MLOps infrastructure.
Benefits
- Work on cutting-edge AI infrastructure and low-latency inference problems.
- Collaborate with world-class engineers and AI researchers.
- High ownership and direct impact in a fast-growing team with minimal bureaucracy.
- Equal-opportunity and inclusive workplace.