Infrastructure Engineer, TL
Arena Intelligence, Inc.2 months ago
Responsibilities
- Build low-latency, high-reliability APIs for leaderboards, models, and arenas.
- Solve SSE and streaming-response challenges across heterogeneous model providers, including partial-failure recovery, mid-stream fallback, and response normalization.
- Build enterprise infrastructure for rate limiting, authentication, usage metering, cost attribution, audit logging, and SOC 2 compliance.
- Implement distributed tracing, latency breakdowns, token-level usage tracking, and real-time dashboards.
- Integrate infrastructure with Arena's evaluation platform, Arena data, and customer-specific benchmarks in collaboration with research teams.
- Contribute to the backend of the Leaderboards and Evals platforms and help unify public and private data architectures.
Requirements
- At least 4 years of backend engineering experience, including meaningful experience with distributed systems, infrastructure, or developer-facing platforms.
- Strong proficiency in Go and/or Rust and hands-on experience building high-throughput APIs or proxy/gateway systems.
- Experience with LLM provider APIs such as OpenAI, Anthropic, and Google, including streaming, token management, rate limits, and model-specific behavior.
- Cloud infrastructure experience with AWS or GCP, Kubernetes, Terraform, and database systems such as Postgres and Redis.
- Product-oriented approach focused on developer experience and comfort working in an ambiguous startup environment.
- Nice-to-have experience with API gateways, proxies, developer tools, ML infrastructure, model serving, or evaluation frameworks.
- Nice-to-have experience with SSO, RBAC, audit logs, multi-tenancy, or billing infrastructure using Stripe, Metronome, and Orb.
- Familiarity with vLLM, LiteLLM, and LangChain is a plus.
Benefits
- Competitive compensation and equity aligned to the team member's market, with base salary depending on permanent work location.
- Comprehensive medical, dental, vision, and additional health and wellness support programs.
- Opportunity to work on cutting-edge AI with a small, mission-driven team.
- Culture emphasizing transparency, trust, and community impact.
Tech Stack
About Arena Intelligence, Inc.
Created by researchers from UC Berkeley, Arena (formerly LMArena) is a community-powered platform to measure and advance the frontier of AI for real-world use. Tens of millions of builders, researchers, and creative professionals come to Arena to use frontier models and give feedback on their responses, shaping a public leaderboard grounded in real-world use.