Software Engineer III/Senior, AI Gateway

about 4 hours ago

Base Salary

$166k - $202k/yr

Responsibilities

Build and evolve the AI Gateway components for real-time AI traffic management.
Design and implement AI Gateway Traffic Policy Objects to enforce usage limits and access rules.
Track AI usage and token consumption to provide insights to customers.
Make AI behavior observable and explainable for customer inquiries.
Design abstractions that simplify user interactions with the gateway.
Collaborate with various teams to ensure alignment of AI usage data and policy enforcement.

Proficiency in a statically typed, compiled language such as Go, Rust, C++, or Java.
Experience working with AI/LLMs and understanding their unique challenges.
Strong focus on developer experience and thoughtful abstractions.
Ability to define system behavior beyond basic plumbing.
Familiarity with retries, limits, and cost management in systems.
Interest in reducing user complexity through system design.

Full premiums covered on base healthcare, dental, and vision for employees.
401(k) retirement plan with matching contributions.
Open and flexible vacation policy.
Parental leave of up to 16 weeks for birthing parents and 8 weeks for new parents.
Annual professional development budget and home office stipend.
Co-working space stipend for remote employees.
Free lunches for employees onsite at the San Francisco office.
Company offsites twice a year for team bonding.
Bi-annual performance reviews for feedback and compensation adjustments.