Software Engineer, LLM Infrastructure
Fireworks AI11 months ago
San Mateo, CA, USAMid Level
H1B sponsor
Base Salary
$175k - $220k/yr
Responsibilities
- Design and develop scalable backend infrastructure for distributed training, inference, and data pipelines.
- Build and maintain LLM CI/CD pipelines, control-plane services, and model-serving systems.
- Improve performance, cost efficiency, and reliability across compute, storage, and networking layers.
- Build frameworks and safeguards to improve model quality.
- Collaborate with performance, training, product, cloud infrastructure, and research teams on infrastructure solutions.
- Participate in code reviews, technical discussions, and continuous integration and deployment processes.
Requirements
- Bachelor’s degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience.
- At least 3 years of software engineering experience focused on infrastructure or machine-learning systems.
- Strong programming skills in Python, Go, or a similar language.
- Experience with ML infrastructure and tooling such as PyTorch, MLflow, Vertex AI, SageMaker, and Kubernetes.
- Basic understanding of LLM concepts including context length, disaggregated prefill, and KV cache memory estimation.
- Preferred: 5+ years of software engineering experience focused on infrastructure or machine-learning systems.
- Preferred: experience with open-source inference engines such as vLLM, Sglang, or TRT-LLM.
- Preferred: contributions to open-source infrastructure or machine-learning projects.
- Preferred: experience building large-scale ML/MLOps infrastructure.
Tech Stack
Categories
About Fireworks AI
Fireworks AI builds a generative AI platform for developers and enterprises to train, fine-tune, and serve open models for production use across text, image, audio, embeddings, and multimodal workloads. It offers managed inference and tooling via APIs on globally distributed infrastructure, with a usage-based SaaS model. Founded in 2022 and headquartered in San Mateo, CA, Fireworks AI is a privately held, Series D company.