2 days ago
Shanghai, China or Beijing, ChinaIntern
Responsibilities
- Analyze and optimize inference performance across model architecture, numerical precision, GPU kernels, memory systems, and end-to-end serving frameworks.
- Develop low-bit quantization techniques for model weights, activations, attention operands, and KV caches.
- Explore FP8, NVFP4, MXFP4, and other low-precision formats and mixed-precision strategies while preserving model accuracy.
- Design and evaluate KV cache compression, sparsification, hierarchical storage, and reuse techniques.
- Optimize inference for MHA, GQA, MQA, MLA, sliding-window attention, sparse attention, linear attention, and state space models.
- Investigate speculative decoding, state reuse, computation reuse, and other model-system co-optimization techniques.
- Build performance models and optimization strategies for long-context inference and agentic workloads.
- Collaborate with model, framework, kernel, library, system software, and GPU architecture teams.
Requirements
- Pursuing an MS or PhD in Computer Science, Computer Engineering, Electrical Engineering, Applied Mathematics, or a related field, with expected graduation in 2028.
- Available to work as an intern for at least three months and at least four days per week.
- Strong understanding of Transformer architectures and large language model inference, including attention, MoE, and KV cache management.
- Experience with at least one large language model inference framework, such as TensorRT-LLM, vLLM, SGLang, or a comparable framework.
- Proficiency in Python and PyTorch.
- Experience with at least one GPU programming technology, such as CUDA C/C++, Triton, or a comparable parallel programming framework.
- Understanding of GPU architecture, memory hierarchy, Tensor Cores, parallel computing, and performance analysis.
- Familiarity with quantization error, scaling, rounding, clipping, group or block quantization, and mixed-precision computation.
- Ability to evaluate optimization techniques across model accuracy, kernel performance, and end-to-end inference performance.
- Strong experimental design, data analysis, debugging, problem-solving, communication, technical writing, organization, and cross-functional collaboration skills.
- Preferred experience includes optimizing kernels or end-to-end workloads on NVIDIA GPUs; research or engineering in model quantization, KV cache optimization, efficient attention, speculative decoding, or long-context inference; contributions to inference frameworks, GPU libraries, compilers, or relevant open-source projects; and profiling production-scale LLM inference workloads.
Benefits
- Internship duration of at least three months.
- Internship schedule requires at least four days per week.
- Expected graduation date is 2028.
Categories
About Nvidia
Nvidia designs and sells GPUs and accelerated computing platforms for data centers, AI/ML, graphics, gaming, and automotive, monetizing through hardware, software platforms (CUDA, AI frameworks), and systems like DGX and networking. Customers include cloud providers, enterprises, researchers, and OEMs. Founded in 1993 and headquartered in Santa Clara, it is a public company traded on NASDAQ under NVDA.