Forward Deployed Engineer, Lead - LLM Post-training
Reflection10 months ago
Responsibilities
- Lead enterprise post-training engagements by assessing customer data, defining training strategies, designing reward signals and verifiers, preparing datasets, running training loops, and evaluating customer-specific benchmarks.
- Build reinforcement learning environments, synthetic data generation pipelines, reward model training systems, and preference data collection workflows.
- Design evaluation infrastructure, including evaluation harnesses, test sets, baselines, and use-case-specific success metrics.
- Own the pipeline from raw customer data through inspection, cleaning, standardization, and training-ready datasets.
- Deploy post-trained models across public cloud, VPC, and on-premises environments while optimizing inference performance, cost, and reliability.
- Define post-training playbooks, best practices, and technical standards; mentor engineers and scale the applied AI practice.
- Work directly with customers and research teams to translate domain requirements into training strategies and model improvements.
Requirements
- Hands-on experience post-training large language models at scale, including RL training environments and preference optimization on models with 50B+ parameters.
- Experience building synthetic data generation pipelines, reward models, verifiers, and reinforcement learning data and feedback loops.
- Deep understanding of evaluation methodology, training dynamics, benchmarks, and real-world model performance.
- Practical experience with multi-node GPU clusters, large training runs, distributed training debugging, and cost optimization.
- Strong software engineering fundamentals, including production-quality code, data pipelines, dataset and model version control, and reproducible workflows.
- At least 6 years of engineering experience, including at least 2 years focused on LLM post-training in a leadership capacity.
- Experience in customer-facing technical roles or a genuine interest in developing customer-facing skills.
- Ability to work with high agency and ownership in a fast-paced startup environment.
Benefits
- Comprehensive medical, dental, vision, life, and disability insurance.
- Fully paid parental leave for all new parents, including adoptive and surrogate journeys.
- Financial support for family planning.
- Paid time off and relocation support.
- Lunch and dinner provided daily.
- Regular off-sites and team celebrations.
Categories
Forward DeployedML Engineering
About Reflection
Reflection is a research lab making intelligence open and accessible for everyone to use, customize, and build on. Our team previously built frontier LLMs at labs like DeepMind, OpenAI, and Anthropic. We believe AI should be built in the open, with transparent research and collaborative development. That means giving enterprises, governments, and sovereign entities true ownership and control of AI that performs at the highest level. Our mission: make intelligence open and accessible to all.