Staff Engineer, Inference Optimizations
DigitalOcean3 months ago
Base Salary
$191k - $239k/yr
Responsibilities
- Lead technical strategy for benchmarking and performance optimization at inference-engine and GPU-kernel layers.
- Optimize attention layers, memory and precision management, and parallel execution across multi-node GPU clusters.
- Tune AMD AITER, composable kernel, assembly, FP8, BF16, FlashAttention, RMS Norm, and mixture-of-experts router kernels for large models.
- Advise on NVIDIA and AMD hardware procurement and software integration.
- Develop and deploy quantization techniques using FP8, INT8, and experimental FP4 to improve throughput while preserving accuracy.
- Provide technical mentorship through code and design reviews without direct administrative management.
- Partner with Product Management and TPMs to turn hardware capabilities and limits into shippable product features.
- Contribute to and integrate open-source work within GPU infrastructure and model-performance optimization communities.
Requirements
- 5+ years of experience in high-performance computing or AI infrastructure, including solving compute-utilization and memory-bandwidth bottlenecks.
- Deep familiarity with generative AI and the architectures of major LLM, VLM, and LMM model families.
- Hands-on experience with attention-layer optimization and parallelization across distributed GPU environments.
- Comprehensive understanding of NVIDIA and AMD GPU architectures and their software ecosystems, including CUDA and ROCm.
- Extensive experience integrating, building with, and contributing to open-source software projects.
- Strong system design skills involving low-level GPU programming, optimization, memory access patterns, and parallel execution.
- Experience acting as a technical lead and driving design and delivery through cross-functional alignment and expert-level delegation.
- Deep understanding of GPU architectures, including SMs, warp scheduling, and Tensor Cores.
- Expert-level Triton or CUDA experience; Triton compiler contributions or custom CUDA kernels for major LLMs are valued.
Benefits
- Remote role.
- Reimbursement for relevant conferences, training, and education.
- Access to LinkedIn Learning with more than 10,000 courses.
- Employee Assistance Program, local employee meetups, and flexible time off.
- Eligible employees may receive bonuses, equity grants upon hire, and access to an Employee Stock Purchase Program.
Categories
About DigitalOcean
DigitalOcean provides cloud infrastructure and platform services for developers, startups, and small to mid-sized businesses, including virtual machines (Droplets), managed Kubernetes and databases, object/block storage, networking, and GPUs for AI workloads. It operates a usage-based, self-service public cloud with APIs, CLI, and a marketplace to deploy and scale applications. Founded in 2012 and headquartered in Broomfield, Colorado, DigitalOcean is a public company listed on the NYSE.