3 days ago
Paris, FranceSenior
Responsibilities
- Design, build, and operate distributed training and inference infrastructure for large language models, including Mixture-of-Experts models.
- Scale training through data, tensor, pipeline, and expert parallelism, memory optimization, checkpointing, and failure recovery.
- Build inference, rollout, evaluation, experimentation, reinforcement-learning, and sandboxed execution systems.
- Own performance validation through benchmarks, throughput and rollout monitoring, and dashboards for regression detection.
- Improve researcher workflows involving job submission, scheduling, quotas, GPU utilization, and observability.
- Partner with platform teams on GPU-cluster capacity, networking, and scheduling across GPU providers.
- Investigate performance and reliability issues across hardware, software, and distributed systems.
Requirements
- Demonstrated depth in distributed systems and ML infrastructure for large-scale training and inference.
- Proficiency in Python and at least one systems language such as Rust, C++, or Go.
- Experience operating large-scale training or serving infrastructure, schedulers, orchestration systems, or other high-performance GPU workloads.
- Understanding of throughput and latency constraints in distributed inference, including communication, memory, scheduling, and stragglers.
- Strong debugging, profiling, and incident-diagnosis skills, ideally including on-call experience.
- Ability to explain design and performance trade-offs to technical and non-technical audiences.
- Prior machine-learning model-training experience is valuable but not required for candidates with strong distributed-systems or HPC backgrounds.
- Familiarity with distributed training frameworks such as Megatron-LM, TorchTitan, or DeepSpeed is a bonus.
- Experience with LLM inference engines such as vLLM or SGLang, reinforcement-learning systems, rollout infrastructure, NCCL, RDMA, low-latency networking, GPU benchmarking, profiling, cluster monitoring, sandboxed code execution, or open-source ML and distributed-systems projects is a bonus.
Benefits
- Competitive global benefits.
- New-hire stock equity through RSUs and an employee stock purchase plan.
- Opportunities to collaborate with colleagues across Datadog offices in New York City and Paris.
- Opportunities to attend and present at conferences and meetups.
- Intra-departmental mentor and buddy program.
- Inclusive culture and access to Community Guilds and employee resource groups.
- Benefits vary by country of employment and nature of employment.
- The posting indicates a hybrid work arrangement via #LI-Hybrid.
Tech Stack
Categories
AI ResearchML Engineering
About Datadog
Datadog builds a SaaS observability and security platform that monitors infrastructure, applications, logs, and services for engineering and DevOps teams. Its products include infrastructure monitoring, APM, log management, real user monitoring, and cloud security, sold via subscriptions and used across cloud-native and hybrid environments. Founded in 2010 and headquartered in New York City, Datadog is a public company trading on NASDAQ under the ticker DDOG.