Site Reliability Engineer
Arena Intelligence, Inc.2 months ago
Responsibilities
- Build and orchestrate API-based products, including low-latency, high-reliability APIs for leaderboards, models, and arenas.
- Build enterprise-grade infrastructure for rate limiting, authentication, usage metering, cost attribution, audit logging, and SOC 2 compliance.
- Instrument infrastructure with distributed tracing, latency breakdowns, token-level usage tracking, and real-time dashboards.
- Integrate with the Arena evaluation platform, Arena data, and customer-specific benchmarks to create AI-centered products.
- Work as an early infrastructure-team member with researchers, engineers, and product leadership on zero-to-one and scale-up initiatives.
Requirements
- 6+ years of backend engineering experience, including meaningful work with distributed systems, infrastructure, or developer-facing platforms.
- Strong proficiency in Go and/or Rust and hands-on experience building high-throughput APIs or proxy/gateway systems.
- Experience with LLM provider APIs such as OpenAI, Anthropic, and Google, including streaming, token management, rate limits, and model-specific behavior.
- Cloud infrastructure experience with AWS or GCP, Kubernetes, Terraform, and database systems such as Postgres and Redis.
- Product-oriented approach focused on developer experience and comfort working in an ambiguous startup environment.
- Preferred experience with API gateways, proxies, or developer tools such as Bifrost, Kong, Envoy, Tyk, or custom systems.
- Preferred background in ML infrastructure, model serving, or evaluation frameworks.
- Preferred experience with SSO, RBAC, audit logs, multi-tenancy, and billing infrastructure using Stripe, Metronome, or Orb.
- Familiarity with modern AI infrastructure tools such as vLLM, LiteLLM, and LangChain.
Benefits
- Competitive compensation and equity aligned to the markets where team members are based, with base salary dependent on permanent work location.
- Comprehensive medical, dental, vision, and additional wellness support programs.
- Opportunity to work on cutting-edge AI with a small, mission-driven team.
- Culture focused on transparency, trust, community impact, diversity, and inclusion.
Tech Stack
About Arena Intelligence, Inc.
Created by researchers from UC Berkeley, Arena (formerly LMArena) is a community-powered platform to measure and advance the frontier of AI for real-world use. Tens of millions of builders, researchers, and creative professionals come to Arena to use frontier models and give feedback on their responses, shaping a public leaderboard grounded in real-world use.