2 days ago
Base Salary
$272k - $431k/yr
Responsibilities
- Design and evolve a unified memory layer spanning GPU memory, pinned host memory, RDMA-accessible memory, SSD tiers, and remote file, object, and cloud storage.
- Architect and implement integrations with vLLM, SGLang, and TensorRT-LLM for KV-cache offload, reuse, and remote sharing.
- Design interfaces and protocols for disaggregated prefill, peer-to-peer KV-cache sharing, and multi-tier KV-cache storage.
- Use GPUDirect, RDMA, NVLink, and related technologies to optimize low-latency access across accelerators and memory pools.
- Profile and optimize CPU, GPU, memory, and network performance while validating improvements in time to first token and throughput.
- Mentor engineers, set technical direction for memory and storage subsystems, and lead cross-functional technical efforts.
- Represent the team in internal reviews, open-source projects, conferences, and customer-facing technical discussions.
Requirements
- Master’s degree, PhD, or equivalent experience.
- 15+ years of experience building large-scale distributed systems, high-performance storage, or ML systems infrastructure in C/C++ and Python, with production delivery experience.
- Deep understanding of GPU HBM, host DRAM, SSD, and remote or object-storage memory hierarchies.
- Experience with distributed caching or key-value systems optimized for low latency and high concurrency.
- Hands-on experience with networked I/O and RDMA, NVMe-oF, or NVLink-style technologies, including disaggregated and aggregated AI-cluster deployments.
- Strong systems profiling and optimization skills across CPU, GPU, memory, and networking, using metrics to guide architecture and validate latency and throughput improvements.
- Excellent communication skills and experience leading cross-functional work with research, product, and customer teams.
- Preferred qualifications include contributions to open-source LLM-serving or systems projects, KV-cache optimization, compression, streaming, or reuse experience; unified memory or storage-layer design experience; and publications or patents in LLM systems, memory-disaggregated architectures, RDMA/NVLink data planes, or ML KV-cache systems.
Benefits
- Competitive base salary of $272,000–$431,250, plus equity and benefits.
- Comprehensive benefits package.
- Applications accepted at least until November 3, 2026.
Categories
About Nvidia
Nvidia designs and sells GPUs and accelerated computing platforms for data centers, AI/ML, graphics, gaming, and automotive, monetizing through hardware, software platforms (CUDA, AI frameworks), and systems like DGX and networking. Customers include cloud providers, enterprises, researchers, and OEMs. Founded in 1993 and headquartered in Santa Clara, it is a public company traded on NASDAQ under NVDA.