3 days ago
Remote, United StatesMid Level
Base Salary
$180k - $260k/yr
Responsibilities
- Build repeatable benchmarks and evaluation pipelines for model accuracy, hallucinations, retrieval quality, latency, and cost.
- Create dashboards that expose system performance changes to the team.
- Turn research prototypes into production retrieval, drafting, document, and agent workflows.
- Develop backend services, APIs, and internal tools supporting testing and releases.
- Deploy and operate cloud infrastructure, including monitoring, databases, and delivery pipelines.
- Collaborate with AI research while staying aligned with product and user needs.
Requirements
- At least 3 years of experience as a software, infrastructure, or product engineer building and deploying production systems.
- Strong Python skills and hands-on experience deploying software to the cloud.
- Experience building evaluation pipelines, benchmarks, or assessment systems for machine-learning or AI systems.
- Experience with APIs, databases, backend services, deployment pipelines, and monitoring or performance dashboards.
- Ability to turn experimental code or ambiguous ideas into usable, maintainable products.
- Full-stack experience is a plus.
- Familiarity with LLMs, retrieval-augmented generation, agents, AI benchmarks, or document automation is valuable.
Benefits
- Remote within the United States.
- Visa sponsorship is not available.