9 hours ago
Remote, United StatesMid Level
Base Salary
$180k - $260k/yr
Responsibilities
- Build repeatable benchmarks and evaluation pipelines for AI system accuracy, hallucinations, retrieval quality, latency, and cost.
- Create dashboards to track changes in system performance.
- Turn AI research prototypes into retrieval, drafting, document, and agent workflows.
- Build backend services, APIs, and internal tools for testing and releasing improvements.
- Deploy and operate cloud infrastructure, including monitoring, databases, delivery pipelines, and reliable operations.
- Collaborate with AI research while incorporating product and user needs.
Requirements
- At least 3 years of experience as a software, infrastructure, or product engineer building and deploying production systems.
- Strong Python skills and hands-on cloud deployment experience.
- Experience building evaluation pipelines, benchmarks, or assessment systems for machine learning or AI systems.
- Experience with APIs, databases, backend services, deployment pipelines, and dashboards or monitoring systems.
- Ability to turn experimental code and ambiguous ideas into maintainable products.
- Interest or experience with LLMs, retrieval-augmented generation, agents, AI benchmarks, or document automation.
- Full-stack experience is a plus.
- Familiarity with FastAPI, Docker, React, Next.js, Postgres, or vector databases is helpful.
Benefits
- Annual base salary is provided based on relevant skills, experience, and role scope.
- Remote within the United States.
- Visa sponsorship is not available.
Tech Stack
Categories
AI ApplicationsProduct Engineering