5 months ago
London, United KingdomSenior
Responsibilities
- Build the xAI API that serves models to developers worldwide.
- Own the end-to-end high-throughput inference system, including model serving infrastructure, request routing, SDK development, rate limiting, observability, and efficient scaling.
- Operate the system for billions of tokens per minute with low latency and high availability.
Requirements
- Expert knowledge of Rust or C++.
- Experience designing, implementing, and maintaining reliable, horizontally scalable distributed systems.
- Knowledge of service observability and reliability best practices.
- Experience operating PostgreSQL, ClickHouse, and MongoDB.
- Experience with LLM inference engines and serving frameworks such as SGLang, TensorRT, or vLLM is preferred.
- Experience designing or building with agent SDKs and agent orchestration frameworks is preferred.
- Experience with Docker, Kubernetes, and containerized applications is preferred.
- Expert knowledge of gRPC, including unary, response streaming, bi-directional streaming, and REST mapping, is preferred.
Tech Stack
Categories
About xAI
Understand the Universe. We are a team of AI technologists and business leaders on a mission to build AI systems that can help humanity understand the world better. https://x.ai/careers