Nvidia

MLOps Senior Lead Engineer

Nvidia
Apply
3 days ago
Seattle, WA, USA or Santa Clara, CA, USAStaff+
H1B sponsor

Base Salary

$188k - $442k/yr

Responsibilities

  • Set the technical direction for the MLOps team and serve as the senior hands-on engineer for complex customer and performance situations.
  • Partner directly with researchers to diagnose training and inference performance problems and recurring crashes across compute, network, and storage.
  • Debug large-scale distributed training involving NCCL, high-speed interconnects, RDMA, parallel storage, containers, Slurm, and Kubernetes.
  • Identify recurring customer problems, coordinate with platform teams and partners, and translate needs into platform feature requests.
  • Run customer workloads and shift platform testing earlier to identify issues before capacity handoff.
  • Create training, guidance, reference tests, blog posts, talks, and papers.
  • Mentor engineers, pair on bring-ups, lead design reviews, and maintain clear written technical decisions and ownership.

Requirements

  • Bachelor’s or master’s degree in computer science, engineering, or a related field, or equivalent experience.
  • At least 8 years designing, operating, or supporting large-scale GPU or HPC compute infrastructure, including distributed training and inference.
  • Hands-on experience with Slurm or Kubernetes and containers such as Docker, Enroot, Apptainer, or Singularity.
  • Experience with distributed training frameworks such as PyTorch, NeMo, Megatron, or JAX, and inference servers such as Triton, TorchServe, or Ray Serve.
  • Experience profiling distributed training with PyTorch profiler or flight recorder, Nsight Systems, and Nsight Compute, with strong knowledge of NCCL collectives.
  • Fluency in Python and at least one systems language such as Go, Rust, or C++, plus familiarity with infrastructure as code such as Terraform.
  • Ability to isolate performance problems across compute, network, and storage and debug complex systems.
  • Preferred qualifications include GPU cluster bring-up or acceptance testing at thousand-node scale, MLPerf or similar benchmarking, InfiniBand, RDMA, Lustre, GPFS, CUDA, OpenTelemetry, Grafana, ClickHouse, goodput or efficiency metrics, and technical leadership with mentoring and partner or vendor engagement.
  • Strong data-driven judgment and excellent written and verbal communication skills are required.

Benefits

  • Comprehensive benefits package for employees and their families.
  • Equity eligibility.
  • Applications are accepted at least until November 23, 2026.
Nvidia

About Nvidia

10,000+ employees

Nvidia designs and sells GPUs and accelerated computing platforms for data centers, AI/ML, graphics, gaming, and automotive, monetizing through hardware, software platforms (CUDA, AI frameworks), and systems like DGX and networking. Customers include cloud providers, enterprises, researchers, and OEMs. Founded in 1993 and headquartered in Santa Clara, it is a public company traded on NASDAQ under NVDA.

Contact me