2 months ago
Shanghai, China or Beijing, ChinaIntern
Responsibilities
- Work directly with application developers to understand workload challenges and build or optimize parallel algorithms and data structures for GPUs.
- Optimize large language model training and inference and contribute to LLM frameworks and open-source projects.
- Collaborate with NVIDIA architecture, research, libraries, tools, and system software teams to improve platform performance and developer efficiency.
- Perform GPU kernel, instruction-level, compiler, operator, framework, and algorithm optimization.
- Improve distributed training and inference communication through communication libraries, interconnect analysis, and compute-communication overlap strategies.
Requirements
- Have a degree or equivalent experience from a university in an engineering or computer science-related field; master's or doctoral candidates are preferred.
- Have solid knowledge of C, C++, Python, or Fortran, software development, programming techniques, and algorithms.
- Have strong mathematical fundamentals, including linear algebra and numerical methods.
- Have a background in parallel programming and accelerated computing, including parallel architectures and performance analysis and tuning.
- Have experience with performance analysis and optimization in large language models or high-performance computing; operator-, framework-, and algorithm-level expertise is preferred.
- Experience with distributed communication optimization, remote direct memory access, GPU interconnects, collective communication algorithms, and large-scale training and inference libraries is highly advantageous.
- Demonstrate software engineering fundamentals, system architecture thinking, and the ability to build modules in complex systems.
- Communicate and collaborate effectively with architecture, research, and software product teams.
Categories
About Nvidia
Nvidia designs and sells GPUs and accelerated computing platforms for data centers, AI/ML, graphics, gaming, and automotive, monetizing through hardware, software platforms (CUDA, AI frameworks), and systems like DGX and networking. Customers include cloud providers, enterprises, researchers, and OEMs. Founded in 1993 and headquartered in Santa Clara, it is a public company traded on NASDAQ under NVDA.