Member of Technical Staff, AI Training Infrastructure
Fireworks AI2 months ago
San Mateo, CA, USA or New York, NY, USAMid Level
H1B sponsor
Base Salary
$200k - $350k/yr
Responsibilities
- Design and implement scalable infrastructure for large-scale model training workloads.
- Develop and maintain distributed training pipelines for LLMs and multimodal models.
- Optimize training performance across multiple GPUs, nodes, and data centers.
- Implement monitoring, logging, and debugging tools for training operations.
- Architect and maintain data storage solutions for large-scale training datasets.
- Automate infrastructure provisioning, scaling, and orchestration for model training.
- Collaborate with researchers to implement and optimize training methodologies.
- Analyze and improve the efficiency, scalability, and cost-effectiveness of training systems.
- Troubleshoot complex performance issues in distributed training environments.
Requirements
- Bachelor’s degree in Computer Science, Computer Engineering, or a related field, or equivalent practical experience.
- At least 3 years of experience with distributed systems and ML infrastructure.
- Experience with PyTorch and proficiency in AWS, GCP, or Azure.
- Experience with Kubernetes and Docker.
- Knowledge of distributed training techniques including data parallelism, model parallelism, and FSDP.
- Master’s or PhD in Computer Science or a related field is preferred.
- Experience training large language models or multimodal AI systems is preferred.
- Experience with ML workflow orchestration tools and high-performance distributed computing is preferred.
- Familiarity with ML DevOps practices and contributions to open-source ML infrastructure are preferred.
Benefits
- Work on cutting-edge AI infrastructure and model training systems.
- Collaborate with experienced engineers, AI researchers, and an ambitious team.
- Inclusive, equal-opportunity workplace.
Categories
About Fireworks AI
Fireworks AI builds a generative AI platform for developers and enterprises to train, fine-tune, and serve open models for production use across text, image, audio, embeddings, and multimodal workloads. It offers managed inference and tooling via APIs on globally distributed infrastructure, with a usage-based SaaS model. Founded in 2022 and headquartered in San Mateo, CA, Fireworks AI is a privately held, Series D company.