5 months ago
San Francisco, CA, USA or New York, NY, USASenior
Responsibilities
- Build and evolve Cursor’s inference gateway as a unified abstraction over provider API semantics.
- Design intelligent cross-provider failover to reduce user-visible degradation during provider outages.
- Design routing backpressure and admission control to prevent traffic spikes from cascading into providers.
- Build and operate production inference systems handling millions of requests.
- Balance reliability, cost, latency, and user experience in large-scale system design decisions.
Requirements
- Deep experience building high-throughput, low-latency distributed systems.
- Particularly strong experience in inference serving, traffic routing, or real-time data pipelines.
- Ability to reason about GPU utilization, provider economics, and capacity planning at scale.
- Strong software engineering fundamentals and experience shipping production systems handling millions of requests.
- Ability to make sound tradeoffs among reliability, cost, latency, and user experience.
Categories
