6 hours ago
Responsibilities
- Build and operate the globally distributed inference system across Relace’s GPU fleet.
- Develop the routing layer that determines which machine handles each request.
- Design behavior for handling demand beyond available capacity.
- Build systems that scale the multi-cloud fleet up and down as demand changes.
- Develop and maintain Rust services and Kubernetes services managing the fleet.
- Own production systems running under load and debug production failures in detail.
Requirements
- Around five or more years building production systems where latency and failure behavior mattered.
- Experience with systems such as serving systems, control planes, schedulers, databases, networking, or trading systems; data pipelines and analytics are not the target background.
- Strong proficiency in a systems language such as Rust, C++, or Go.
- Demonstrated ownership of a production system running under load.
- Ability to explain a production failure personally debugged in detail.
- Kubernetes experience at the level of having extended it, particularly for low-reliability infrastructure.
- Experience with a scheduler or load balancer as a primary project is preferred.
- Experience with spot or preemptible capacity is preferred.
- Exposure to AI serving systems is preferred but optional.
Benefits
- Competitive compensation plus equity.
- San Francisco-based role requiring in-person work five days a week.
Tech Stack
Categories
