2 months ago
Base Salary
$165k - $225k/yr
Responsibilities
- Train large byte-native and multimodal foundation models across massive, heterogeneous corpora.
- Implement and evaluate model architectures, training objectives, and optimization methods.
- Develop stable pre-training recipes and conduct scaling experiments for novel architectures.
- Perform ablations and analyze training dynamics, model behavior, metrics, and base-model quality.
- Build production-grade training code and collaborate with data and distributed-training engineers to improve efficiency, reliability, and scalability.
- Own or contribute to end-to-end training and evaluation pipelines with monitoring and reproducibility.
Requirements
- 5+ years of experience in machine learning research or engineering.
- Proven experience developing and pre-training large language or multimodal foundation models.
- Strong general software engineering skills and the ability to write robust, performant training code.
- Solid understanding of deep learning fundamentals, modern pre-training methods, and related literature.
- Ability to implement research ideas and evaluate them using baselines, ablations, metrics, and analysis.
- Hands-on experience running GPU-based training workloads and familiarity with distributed training.
- MS or PhD in Computer Science, Machine Learning, Artificial Intelligence, Mathematics, or a related field.
- Preferred experience with JAX, Flax, and XLA.
- Preferred experience with multi-node pre-training using FSDP, ZeRO, or Megatron.
- Preferred experience developing training recipes, ablations, scaling experiments, and reproducible monitored training and evaluation pipelines.
Benefits
- Medical, dental, and vision insurance.
- 401k plan.
- Daily lunch, snacks, and beverages.
- Flexible time off.
- Competitive salary and equity.
Categories
AI ResearchML Engineering
