1 day ago
Base Salary
$224k - $357k/yr
Responsibilities
- Analyze distributed training and inference performance across the model, framework, data-loading, checkpointing, compiler, kernel, CUDA, NCCL, host, and network layers.
- Identify performance bottlenecks, validate root causes with supporting evidence, and recommend or implement solutions.
- Define fleet-wide GPU performance metrics and determine the signals needed for engineering diagnosis.
- Investigate performance regressions across workloads, machines, software versions, and configurations, and develop reusable analysis methods.
- Evaluate automated diagnoses, identify inaccurate or incomplete findings, and create labeled cases for validation.
- Partner with build partners on real workloads, carry findings through to improvements, and communicate product gaps to the roadmap.
Requirements
- Bachelor’s degree in Computer Science, Computer Engineering, or a related field, or equivalent experience.
- 12+ years of relevant software engineering experience, including hands-on production optimization of GPU training or inference workloads.
- Demonstrated performance analysis across multiple layers of the software stack with documented baselines, evidence, alternative explanations, and validated outcomes.
- Experience diagnosing performance across many jobs, machines, or tenants rather than only tuning a single workload.
- Familiarity with framework, runtime, and compiler interfaces sufficient to determine where signals should be measured.
- Ability to distinguish symptoms from causes, account for noisy or incomplete measurements, and assess when evidence is insufficient.
- Strong programming and debugging skills demonstrated through maintainable analysis code, tests, or repeatable investigation procedures.
- Clear communication of findings, trade-offs, and remaining uncertainty.
- Proficiency with Python and CUDA.
- Preferred experience improving both training and inference, deep knowledge of PyTorch, JAX, Megatron, NCCL, TensorRT-LLM, vLLM, SGLang, or Dynamo, and experience across application, system software, driver, or hardware roles.
- Preferred experience applying statistical judgment, counter-factual analysis, workload segmentation, and building debugging or performance-analysis tools used by other engineers.
Benefits
- Base salary range is 224,000 USD - 356,500 USD, with eligibility for equity and benefits.
- The role is based in Santa Clara, California.
- Applications will be accepted at least until November 2, 2026.
Categories
About Nvidia
Nvidia designs and sells GPUs and accelerated computing platforms for data centers, AI/ML, graphics, gaming, and automotive, monetizing through hardware, software platforms (CUDA, AI frameworks), and systems like DGX and networking. Customers include cloud providers, enterprises, researchers, and OEMs. Founded in 1993 and headquartered in Santa Clara, it is a public company traded on NASDAQ under NVDA.