Forward Deployed Engineer, Lead - LLM Post-training
Reflection11 months ago
Responsibilities
- Lead enterprise post-training engagements by assessing customer data, defining training strategies, designing reward signals and verifiers, preparing datasets, running training loops, and evaluating customer-specific benchmarks.
- Build reinforcement learning environments, synthetic data generation pipelines, reward model training systems, and preference data collection workflows.
- Design evaluation infrastructure, including evaluation harnesses, test sets, baselines, and use-case-specific success metrics.
- Own the pipeline from raw customer data through inspection, cleaning, standardization, and training-ready datasets.
- Deploy post-trained models across public cloud, VPC, and on-premises environments while optimizing inference performance, cost, and reliability.
- Define post-training playbooks, best practices, and technical standards; mentor engineers and scale the applied AI practice.
- Work directly with customers and research teams to translate domain requirements into training strategies and model improvements.
Requirements
- Hands-on experience post-training large language models at scale, including RL training environments and preference optimization on models with 50B+ parameters.
- Experience building synthetic data generation pipelines, reward models, verifiers, and reinforcement learning data and feedback loops.
- Deep understanding of evaluation methodology, training dynamics, benchmarks, and real-world model performance.
- Practical experience with multi-node GPU clusters, large training runs, distributed training debugging, and cost optimization.
- Strong software engineering fundamentals, including production-quality code, data pipelines, dataset and model version control, and reproducible workflows.
- At least 6 years of engineering experience, including at least 2 years focused on LLM post-training in a leadership capacity.
- Experience in customer-facing technical roles or a genuine interest in developing customer-facing skills.
- Ability to work with high agency and ownership in a fast-paced startup environment.
Benefits
- Comprehensive medical, dental, vision, life, and disability insurance.
- Fully paid parental leave for all new parents, including adoptive and surrogate journeys.
- Financial support for family planning.
- Paid time off and relocation support.
- Lunch and dinner provided daily.
- Regular off-sites and team celebrations.
Categories
Forward DeployedML Engineering
About Reflection
Reflection is a New York–based, privately held research lab developing open foundational AI models and agentic coding tools for developers, enterprises, and public-sector users. The team includes former researchers from DeepMind, OpenAI, and Anthropic, and their work focuses on transparent, customizable systems that organizations can deploy with ownership and control.