14 hours ago
Base Salary
$200k - $375k/yr
Responsibilities
- Architect and implement distributed model-serving infrastructure for load balancing, auto-scaling, batch scheduling, and global KV caching.
- Optimize model inference latency, throughput, concurrency, reliability, and tail latency at production scale.
- Benchmark, fine-tune, and accelerate inference engines through GPU kernel work and code generation.
- Develop tools to trace, replay, and resolve issues across orchestration and GPU-kernel layers.
- Build CI/CD infrastructure for endpoint deployment, image publishing, and inference engine updates.
- Advance research on test-time compute scaling, RL rollout, and model-hardware co-design.
Requirements
- Bachelor’s degree in computer science, computer engineering, or a science discipline.
- 5+ years of low-level systems programming experience in C/C++ or Rust.
- Experience with large-scale, highly concurrent production serving.
- Experience with GPU inference engines such as vLLM, SGLang, Triton, or TensorRT-LLM.
- Strong background in batching, caching, load balancing, parallelism, GPU kernels, and code generation.
- Experience with quantization, speculative decoding, distillation, low-precision numerics, testing, benchmarking, reliability, and CI/CD for inference systems.
Benefits
- $200,000–$375,000 base salary, plus equity.
- Comprehensive medical, vision, and dental coverage.
- 401(k) retirement plan.
- Short- and long-term disability insurance and life insurance.
- Various discounts and perks.
Categories
About xAI
Understand the Universe. We are a team of AI technologists and business leaders on a mission to build AI systems that can help humanity understand the world better. https://x.ai/careers