AI Evaluation Engineer
Judi Healthabout 3 hours ago
Charlotte, NC, USA +2 moreMid Level / Senior
Base Salary
$135k - $200k/yr
Responsibilities
- Build and maintain ETL pipelines for diverse data sources.
- Develop data evaluation pipelines to assess production conversations.
- Define benchmarks for accuracy, reliability, and safety signals.
- Design a standardized evaluation framework for multiple agent types.
- Create APIs and internal tools for data scientists and engineers.
- Implement dashboards and monitoring tools for AI quality metrics.
- Collaborate with cross-functional teams to align research and product goals.
Requirements
- 4+ years of experience in data engineering, ML engineering, or software engineering.
- Bachelor’s or Master’s degree in Computer Science, Machine Learning, or a related field.
- Strong proficiency in Python and SQL.
- Experience building and maintaining production data pipelines.
- Familiarity with at least one cloud platform, preferably AWS.