7 days ago
Shanghai, China or Beijing, ChinaIntern
H1B Sponsor
Responsibilities
- Work directly with application developers to understand workload challenges and build or optimize parallel algorithms and data structures for GPUs.
- Optimize large language model training and inference and contribute to LLM frameworks and open-source projects.
- Collaborate with NVIDIA architecture, research, libraries, tools, and system software teams to improve platform performance and developer efficiency.
- Perform GPU kernel, instruction-level, compiler, operator, framework, and algorithm optimization.
- Improve distributed training and inference communication through communication libraries, interconnect analysis, and compute-communication overlap strategies.
Requirements
- Have a degree or equivalent experience from a university in an engineering or computer science-related field; master's or doctoral candidates are preferred.
- Have solid knowledge of C, C++, Python, or Fortran, software development, programming techniques, and algorithms.
- Have strong mathematical fundamentals, including linear algebra and numerical methods.
- Have a background in parallel programming and accelerated computing, including parallel architectures and performance analysis and tuning.
- Have experience with performance analysis and optimization in large language models or high-performance computing; operator-, framework-, and algorithm-level expertise is preferred.
- Experience with distributed communication optimization, remote direct memory access, GPU interconnects, collective communication algorithms, and large-scale training and inference libraries is highly advantageous.
- Demonstrate software engineering fundamentals, system architecture thinking, and the ability to build modules in complex systems.
- Communicate and collaborate effectively with architecture, research, and software product teams.
Categories
About Nvidia
Since its founding in 1993, NVIDIA (NASDAQ: NVDA) has been a pioneer in accelerated computing. The company’s invention of the GPU in 1999 sparked the growth of the PC gaming market, redefined computer graphics, ignited the era of modern AI and is fueling the creation of the metaverse. NVIDIA is now a full-stack computing company with data-center-scale offerings that are reshaping industry.