16 days ago
Seattle, WA, USA or San Francisco, CA, USAMid Level
H1B sponsor
Base Salary
$266k - $445k/yr
Responsibilities
- Design reliable, reproducible, and extensible evaluation pipelines.
- Build continuous evaluation monitoring frameworks, including regression and drift monitoring, golden datasets, and feedback loops.
- Design, build, and maintain backend services and APIs for intelligent automation and knowledge systems.
- Integrate and structure data for downstream systems and AI workflows.
- Collaborate with Data Science, Research, and Engineering teams to integrate OpenAI models into high-leverage workflows.
- Own the full development lifecycle of backend systems and internal platform capabilities.
- Build scalable, maintainable systems while iterating rapidly on new ideas.
Requirements
- At least 4 years of backend engineering experience at product-driven companies, excluding internships.
- Proficiency with Python, FastAPI, and Postgres.
- Experience designing and scaling distributed systems, APIs, or data processing pipelines.
- Experience building AI agents or applications, designing evaluations, and improving performance through prompting or scaffolding.
- Familiarity with LLM evaluation methods, multi-agent workflows, tool use, or long-context patterns.
- Experience creating production evaluations and/or measuring ML or LLM model performance at scale.
- A pragmatic approach to iterative delivery and long-term system development.
Tech Stack
Categories
About OpenAI
OpenAI builds and deploys large-scale AI models and tools—including ChatGPT, GPT-4–class models, DALL·E, and Whisper—sold via APIs and enterprise subscriptions to developers and businesses. It monetizes through usage-based API pricing and ChatGPT Plus/Team/Enterprise, and also reaches customers via Microsoft’s Azure OpenAI Service. Founded in 2015 and headquartered in San Francisco, it operates as a private partnership.