22 hours ago
Remote, United StatesMid Level
Base Salary
$180k - $260k/yr
Responsibilities
- Build repeatable benchmarks and evaluation pipelines for AI model accuracy, hallucinations, retrieval quality, latency, and cost.
- Create dashboards that expose changes in system performance.
- Develop production retrieval, drafting, document, and agent workflows from research prototypes.
- Build backend services, APIs, and internal tools for testing and releasing improvements.
- Deploy and operate cloud infrastructure, including monitoring, databases, delivery pipelines, and reliable operations.
- Collaborate with AI research while remaining connected to product and user needs.
Requirements
- At least 3 years of experience as a software, infrastructure, or product engineer building and deploying production systems.
- Strong Python skills and hands-on cloud deployment experience.
- Experience with APIs, databases, backend services, deployment pipelines, and production monitoring.
- Experience building evaluation pipelines, benchmarks, or assessment systems for machine learning or AI systems.
- Ability to turn ambiguous ideas and experimental code into usable, maintainable products.
- Interest or experience in LLMs, retrieval-augmented generation, agents, AI benchmarks, or document automation.
- Full-stack experience is helpful.
- Familiarity with FastAPI, Docker, React, Next.js, Postgres, or vector databases is a plus.
Benefits
- Annual base salary is $180,000 to $260,000 USD, depending on skills, experience, and role scope.
- Remote within the United States.
- Visa sponsorship is not available.