Member of Technical Staff - Pre-Training Infra
Reflection6 months ago
Responsibilities
- Build and scale distributed training systems for frontier model pre-training.
- Design and operate large-scale training runs with research teams.
- Develop infrastructure for efficient training across thousands of GPUs.
- Optimize training throughput, stability, communication, memory usage, and GPU utilization.
- Build and maintain pipelines for large-scale datasets, checkpointing, and experiment iteration.
- Debug performance bottlenecks involving model parallelism, GPU communication, and training runtime systems.
- Develop systems that enable rapid experimentation with new training techniques.
Requirements
- Experience building or operating distributed training systems for large machine learning models.
- Strong experience with distributed training frameworks such as Megatron and DeepSpeed or similar systems.
- Familiarity with data, tensor, pipeline, or expert parallelism strategies.
- Experience optimizing training throughput and GPU utilization in large distributed environments.
- Familiarity with NCCL and performance tuning for distributed workloads.
- Experience working with ML researchers to productionize experimental training workflows.
- Strong debugging skills across GPU compute, distributed training systems, and large-scale ML pipelines.
- Experience working with large datasets and training pipelines for foundation-model pre-training.
Benefits
- Top-tier compensation and equity.
- Comprehensive medical, dental, vision, life, and disability insurance.
- Fully paid parental leave, including adoptive and surrogate journeys, plus financial support for family planning.
- Paid time off and relocation support.
- Daily lunch and dinner, regular off-sites, and team celebrations.
Categories
About Reflection
Reflection is a New York–based, privately held research lab developing open foundational AI models and agentic coding tools for developers, enterprises, and public-sector users. The team includes former researchers from DeepMind, OpenAI, and Anthropic, and their work focuses on transparent, customizable systems that organizations can deploy with ownership and control.