Staff Software Engineer, GPU Inference
Cerebras Systems4 days ago
Responsibilities
- Productionize and maintain the complete GPU prefill path across API services, model-serving workers, vLLM, PyTorch, ROCm, GPU nodes, networking, and rack-scale infrastructure.
- Establish deployment, upgrade, rollback, health-checking, capacity-management, failure-recovery, and compatibility practices for the AMD GPU fleet.
- Define service-level indicators and objectives and improve fault isolation, graceful degradation, automated recovery, incident response, and remediation.
- Profile and optimize time to first token, throughput, tail latency, GPU utilization, memory efficiency, and rack-level capacity.
- Improve scheduling, continuous batching, prefix caching, KV-cache management, tensor and expert parallelism, request admission, quantization, graph execution, and distributed communication.
- Debug failures and performance regressions across application code, serving frameworks, PyTorch, ROCm/HIP, communication libraries, kernels, drivers, firmware, networking, and hardware.
- Build numerical validation, regression, benchmarking, workload replay, profiling, release qualification, dashboard, and performance-regression infrastructure.
Requirements
- 8+ years of software engineering experience with substantial individual-contributor ownership of complex production systems.
- Experience building, operating, or optimizing production inference systems for large language models, multimodal models, or similarly demanding GPU workloads.
- Strong C++ and Python programming skills, including multithreading, concurrency, memory management, and performance-sensitive software.
- Hands-on experience with a high-performance model-serving framework such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, or an equivalent internal system.
- Strong understanding of GPU execution and performance, including asynchronous execution, memory movement, synchronization, kernel launches, communication overhead, and profiling.
- Experience debugging distributed systems across multiple layers and operating latency-sensitive services in production with Linux, containers, Kubernetes or comparable orchestration, observability, and CI/CD.
- Ability to design rigorous benchmarks, interpret noisy performance results, identify bottlenecks, and translate findings into production improvements.
- Strong communication and technical leadership skills with the ability to drive ambiguous cross-functional projects to completion.
- Bachelor’s degree in Computer Science, Computer Engineering, Electrical Engineering, or a related discipline, or equivalent practical experience.
- Preferred qualifications include AMD Instinct and ROCm experience; CUDA experience; contributions to vLLM, SGLang, PyTorch, Triton, or TensorRT-LLM; distributed and disaggregated inference expertise; multi-GPU and multi-node systems; GPU kernel optimization; reduced-precision inference and quantization; and numerical-validation or performance-regression systems experience.
Benefits
- Opportunity to build a breakthrough AI platform beyond traditional GPU constraints.
- Opportunity to publish and open source cutting-edge AI research.
- Work on one of the fastest AI supercomputers in the world.
- Job stability with startup vitality.
- Simple, non-corporate work culture that respects individual beliefs.
- Equal-opportunity workplace committed to diversity, inclusion, continuous learning, growth, and team support.
Tech Stack
Categories
About Cerebras Systems
Cerebras Systems designs and sells AI compute systems built around its wafer-scale WSE-3 processor, delivered as the CS-3 appliance and via the Cerebras Cloud. It targets enterprises, model labs, and government users needing fast training and inference, and offers on‑prem and cloud deployments. Privately held and headquartered in Sunnyvale, California, the company announced a multi-year partnership with OpenAI to deploy large-scale inference capacity.