3 days ago
Base Salary
$220k - $357k/yr
Responsibilities
- Architect and implement distributed model-serving infrastructure for load balancing, auto-scaling, batch scheduling, and global KV caching.
- Optimize model-inference latency, throughput, tail latency, and reliability at production scale.
- Build and operate high-concurrency serving systems for millions of users.
- Benchmark, fine-tune, and accelerate inference engines, including GPU kernel and code-generation work.
- Develop tools to trace, replay, and resolve issues across orchestration and GPU-kernel layers.
- Create CI/CD infrastructure for endpoint deployment, image publishing, and inference-engine updates.
- Accelerate research on test-time compute scaling, RL rollout, and model-hardware co-design.
Requirements
- Bachelor’s degree in computer science, computer engineering, or a science discipline.
- 5+ years of low-level systems programming experience in C/C++ or Rust.
- Experience with large-scale, high-concurrency production serving is preferred.
- Experience with GPU inference engines such as vLLM, SGLang, Triton, or TensorRT-LLM is preferred.
- Strong background in batching, caching, load balancing, and parallelism is preferred.
- Experience with GPU kernels, code generation, quantization, speculative decoding, distillation, and low-precision numerics is preferred.
- Experience testing, benchmarking, and ensuring the reliability of inference services is preferred.
- Experience designing and implementing CI/CD infrastructure for inference is preferred.
Benefits
- Equity and comprehensive medical, vision, and dental coverage.
- 401(k) retirement plan, short- and long-term disability insurance, life insurance, discounts, and other perks.
Categories
About xAI
Understand the Universe. We are a team of AI technologists and business leaders on a mission to build AI systems that can help humanity understand the world better. https://x.ai/careers