AI Data Engineer
Veeva Systems
3 months ago
Toronto, Canada
Mid Level / Senior
H1B Sponsor
Responsibilities
- Define and establish comprehensive evaluation strategies for new AI Agents.
- Evaluate the quality of LLM-generated content against predefined metrics.
- Design, curate, and generate diverse, high-quality test data sets.
- Develop and maintain scalable automated evaluations for agent behavior.
- Conduct root cause analysis of identified defects or performance issues.
- Document and communicate performance metrics and validation results.
Requirements
- Strong understanding of data quality principles and validation methods.
- Demonstrated skill in advanced prompt engineering techniques.
- Proficiency in designing and deploying automated evaluation pipelines.
- Comfortable with debugging agentic systems and tracing internal reasoning.
- Proficiency in Python and familiarity with test automation tools.
- Bachelor's degree in Data Science, Machine Learning, or related field.
Benefits
- Medical, dental, vision, and basic life insurance.
- PTO and company-paid holidays.
- Retirement programs.
- 1% charitable giving program.
Tech Stack
pytestPython
Categories
AI & MLData EngineeringData ScienceTesting