4 days ago
Remote, United StatesMid Level
Base Salary
$180k - $260k/yr
Responsibilities
- Build repeatable benchmarks and evaluation pipelines for model accuracy, hallucinations, retrieval quality, latency, and cost.
- Create dashboards that visualize system performance changes.
- Develop production retrieval, drafting, document, and agent workflows from research prototypes.
- Build backend services, APIs, and internal tools to test and release improvements.
- Deploy and operate cloud infrastructure, including monitoring, databases, and delivery pipelines.
- Collaborate with AI research while remaining connected to product and user needs.
Requirements
- At least 3 years of experience as a software, infrastructure, or product engineer building and deploying production systems.
- Strong Python skills and hands-on experience deploying software to the cloud.
- Experience building evaluation pipelines, benchmarks, or assessment systems for machine learning or AI systems.
- Experience with APIs, databases, backend services, deployment pipelines, and systems that turn experimental code into maintainable products.
- Experience building dashboards or monitoring systems to visualize performance and operating cloud deployments reliably.
- Full-stack experience is a plus.
- Familiarity with FastAPI, Docker, React, Next.js, Postgres, vector databases, LLMs, retrieval-augmented generation, agents, AI benchmarks, or document automation is helpful.
Benefits
- Annual base salary of $180,000 to $260,000 USD.
- Remote within the United States.
- Visa sponsorship is not available.