2 days ago
Base Salary
$204k - $306k/yr
Responsibilities
- Optimize inference and production serving for latency, throughput, batching, caching, and scheduling.
- Write and tune CUDA and Triton GPU kernels, including kernel fusion, memory and bandwidth optimization, and FP8/INT8 execution.
- Optimize training throughput and GPU utilization through parallelism, communication/compute overlap, mixed precision, and pipeline-stall reduction.
- Build performance models, profiling workflows, and observability for throughput, latency, cost, utilization, and tradeoffs.
- Maintain numerical correctness across precision, kernel, and hardware changes.
- Partner with researchers to productionize models and accelerate experimentation.
- Contribute to distributed training and inference systems where needed, with a primary focus on maximizing GPU performance.
Requirements
- Strong foundations in performance engineering, including profiling, roofline analysis, latency and throughput optimization, and root-cause investigation.
- Deep GPU programming and optimization experience with CUDA and/or Triton, including kernel tuning, memory hierarchy, and bandwidth optimization.
- Hands-on experience optimizing large-model inference and serving, including batching, KV or prompt caching, quantization, and low-latency sampling.
- Hands-on experience optimizing training performance, including parallelism, distributed communication, mixed or low precision, and utilization.
- Working knowledge of PyTorch and/or JAX internals and compiler paths such as torch.compile or XLA.
- Strong Python proficiency and ability to work in C++/CUDA, Rust, or Go as needed.
- Experience at an AI lab or ML-native company and with productionizing research code is preferred.
- Experience with FP8/INT8 quantization, mixed precision, numerical regression detection, large-scale distributed systems, NCCL, NVLink, model or tensor parallelism, and fault tolerance is preferred.
- Experience serving generative, diffusion, video, or 3D/spatial models is preferred.
- Multi-accelerator experience with GPUs, TPUs, or Trainium and experience partnering with hardware vendors is preferred.
- Experience building GPU performance-modeling and observability frameworks is preferred.
- BS, MS, or PhD in Electrical Engineering, Computer Science, Computer Engineering, or a closely related field.
Benefits
- AMD benefits are described in the company's benefits-at-a-glance materials.
About AMD
AMD designs and sells CPUs, GPUs, and adaptive/embedded computing products for PCs, data centers, gaming, and edge devices. Its portfolio includes Ryzen and EPYC processors, Radeon and Instinct graphics, and adaptive SoCs from its Xilinx acquisition, sold to OEMs, cloud providers, and device makers. Founded in 1969 and headquartered in Santa Clara, it is a public company on NASDAQ and supplies semi-custom chips for major game consoles.
