2 days ago
Base Salary
$130k - $225k/yr
Responsibilities
- Lead systems that evaluate tasks across reinforcement learning environments, synthetic data, benchmarks, and domain-specific workflows.
- Set data quality strategy by defining standards, building quality-control systems, and designing experiments to assess agent outputs.
- Develop scalable synthetic data validation methods, including failure-mode analysis, task mutation checks, and trajectory audits.
- Collaborate with research engineers, subject-matter experts, and data vendors to identify quality issues and improve data-generation workflows.
- Translate research insights into production tools, dashboards, validation pipelines, and feedback loops.
- Lead technical projects and mentor research engineers while promoting rigorous, clear, and efficient technical work.
Requirements
- At least 5 years of relevant research or engineering experience, including experience building AI or machine-learning data evaluation systems.
- Experience leading technical projects or teams from problem definition through implementation and iteration.
- Advanced proficiency in Python, Docker, and Linux, with a technical academic or professional background.
- Strong understanding of AI evaluations, post-training, and the qualities of realistic, learnable, diverse, reliable, and useful agent-training data.
- Experience with synthetic data validation, evaluation infrastructure, benchmarks, metrics, experiments, and quality-control workflows.
- Ability to translate research into production systems and scale domain experts' judgment into review or generation processes.
- Clear written communication, attention to detail, independence, and effectiveness in an early-stage startup environment.
Benefits
- Base salary of $130,000 to $225,000 annually.
- Visa sponsorship is available.
- On-site role in the United States.
