Site Reliability Engineer, Provider Operations
OpenRouter7 hours ago
Remote, United StatesMid Level
Responsibilities
- Build and own monitoring for provider and endpoint latency, throughput, error rates, uptime, and output correctness.
- Define provider-tier SLOs, actionable alerts, and observability practices using metrics, tracing, and logs.
- Improve degraded-endpoint detection and automated traffic failover with the routing team.
- Lead on-call incident response, including triage, mitigation, partner communication, postmortems, and follow-up closure.
- Create provider scorecards, SLO reporting, continuous canaries, and evaluations for silent quality regressions.
- Automate endpoint management, capacity changes, deprecations, and rate-limit tuning with safe, auditable tools.
- Build load-testing and launch-readiness tooling for high-volume endpoint traffic.
Requirements
- 4+ years of experience in SRE, production engineering, or infrastructure roles running high-traffic, customer-facing systems.
- Strong experience with observability practices and tooling, including metrics, tracing, logs, SLOs, error budgets, and actionable alerting.
- Software engineering ability with TypeScript and/or Python, with a preference for building tools instead of executing manual runbooks.
- Experience with distributed-systems failure modes such as timeouts, retries, backpressure, partial outages, and noisy neighbors.
- Ability to act as a calm, clear incident commander and communicate effectively with external partners under pressure.
- Understanding of, or willingness to learn, LLM inference serving concepts including streaming, tool calling, prompt caching, throughput and latency tradeoffs, and provider API differences.
- Preferred experience at an inference provider, model lab, GPU cloud, API gateway, or CDN company.
- Preferred experience with routing, load balancing, traffic management, evaluations, or synthetic monitoring for ML systems.
- Experience with TypeScript, Cloudflare Workers, Postgres, ClickHouse, GCP, or Vercel is a plus.
Tech Stack
Categories
Site Reliability
About OpenRouter
OpenRouter builds a unified API and routing layer that lets developers and enterprises access, compare, and manage large language models across multiple providers, with tooling for chat, evals, and usage controls. The company monetizes through metered API usage and enterprise plans that add governance and reliability features. Founded in 2023 and headquartered in New York, it supports hundreds of third-party models and an active developer community on Discord.