AI Data Engineer
Veeva Systems
3 months ago
Boston, MA, USA
Mid Level / Senior
H1B Sponsor
Base Salary
$85k - $225k/yr
Responsibilities
- Define and establish comprehensive evaluation strategies for new AI Agents.
- Evaluate the quality of LLM-generated content against predefined metrics.
- Design, curate, and generate diverse, high-quality test data sets.
- Develop and maintain scalable automated evaluations for agent behavior.
- Conduct root cause analysis of identified defects or performance issues.
- Document and communicate performance metrics and validation results.
Requirements
- Strong understanding of data quality principles and validation methods.
- Demonstrated skill in advanced prompt engineering techniques.
- Proficiency in designing and deploying automated evaluation pipelines.
- Comfortable with debugging agentic systems and tracing internal reasoning.
- Proficiency in Python and familiarity with standard test automation tools.
- Bachelor's degree in Data Science, Machine Learning, Computer Science, or related field.
Benefits
- Medical, dental, vision, and basic life insurance.
- Flexible PTO and company paid holidays.
- Retirement programs.
- 1% charitable giving program.
Tech Stack
pytestPython
Categories
AI & MLData EngineeringData ScienceTesting