4 days ago
Base Salary
$184k - $288k/yr
Responsibilities
- Design and implement JAX program transformations for pipeline schedules, stage placement, sharding, and communication-computation overlap.
- Develop optimizing compiler and runtime technologies for deep learning training on large-scale distributed systems.
- Compile graph intermediate representations to target-specific code and build distributed runtimes that dispatch programs across GPUs.
- Advance compiler technology for large-scale training workloads, including mixture-of-experts models.
- Collaborate with JAX, XLA, hardware architecture, and training-framework teams, including integrating JaxPP with JAX frameworks such as MaxText.
- Contribute to the open-source JaxPP project through release compatibility, testing, performance tracking, and adoption support.
Requirements
- Master's or PhD in Computer Science, Computer Engineering, or a related field, or equivalent experience.
- 6+ years of relevant work or research experience in compilers, distributed systems, or deep learning training.
- Experience designing and implementing compilers and distributed systems.
- Experience with deep learning frameworks such as JAX and PyTorch.
- Experience training and running inference for deep learning models on GPUs.
- Proficiency in C/C++ and Python, including software design, debugging, performance analysis, and test design.
- Experience with parallel computing libraries such as MPI.
- Ability to work independently, define project goals and scope, and lead development efforts.
- Strong communication skills and experience working with partner engineers to debug and tune large training runs.
- Preferred experience with JAX internals, XLA, StableHLO, MLIR, large-scale parallel training, MaxText, Megatron-LM, NCCL, NVSHMEM, RDMA, CPU or GPU architecture, CUDA, and open-source compiler or machine learning projects.
- Mentoring experience with early-career engineers or interns is a bonus.
Benefits
- Competitive salary and a generous benefits package.
- Eligibility for equity and benefits.
Categories
About Nvidia
Nvidia designs and sells GPUs and accelerated computing platforms for data centers, AI/ML, graphics, gaming, and automotive, monetizing through hardware, software platforms (CUDA, AI frameworks), and systems like DGX and networking. Customers include cloud providers, enterprises, researchers, and OEMs. Founded in 1993 and headquartered in Santa Clara, it is a public company traded on NASDAQ under NVDA.