Research Engineer, Forge
Mistral AI2 months ago
Paris, FranceMid Level
Responsibilities
- Build and improve post-training and evaluation workflows for CPT, SFT, RL, and distillation.
- Develop tools and pipelines for synthetic data generation, data curation, training, evaluation, and deployment.
- Debug and harden large-scale ML systems, including distributed training, scheduling, checkpointing, observability, and reproducibility.
- Improve the Forge codebase through clear APIs, tests, documentation, and maintainable abstractions.
- Advance high-throughput asynchronous rollout and scalable post-training systems for frontier-model-scale RL training.
- Make Forge deployment adaptable across customer hardware, software stacks, cloud environments, and on-premises infrastructure.
- Partner with researchers and infrastructure engineers to translate bottlenecks into system improvements.
Requirements
- Strong Python engineering skills and experience working in large codebases, including testing, code review, and operational ownership.
- Hands-on experience with PyTorch, JAX, or similar frameworks.
- Strong systems and infrastructure fundamentals.
- Experience with LLM training or post-training, including fine-tuning, RL, distillation, evaluation, or data pipelines.
- Strong debugging skills for distributed jobs, data issues, quality regressions, and infrastructure failures.
- Clear communication with technical and non-technical stakeholders.
- Preferred experience includes distributed training with FSDP, DeepSpeed, or Megatron.
- Preferred experience includes cluster or orchestration technologies such as SLURM, Ray, Kubernetes, Kueue, Karpenter, or Skypilot.
- Preferred qualifications include reliable ML infrastructure, evaluation systems, large-scale data processing, LLM or agent research, open-source contributions, publications, internal tooling, and training multi-billion-parameter models on petabyte- or exabyte-scale datasets.
Benefits
- Benefits may include healthcare coverage, parental leave, retirement plans, relocation support, wellness programs, and meal and transportation allowances, depending on location.
- Location-specific benefits details are provided on the company Benefits page.
Tech Stack
Categories
About Mistral AI
Mistral AI builds foundation language models and full‑stack AI solutions for enterprises, offering APIs, on‑prem deployment, developer tools, and applications. The privately held company, founded in 2023 and headquartered in Paris, partners with organizations in finance, manufacturing, defense, healthcare, and the public sector to co‑create customized systems. It releases open‑source models alongside commercial offerings and makes its models available through major cloud marketplaces.