16 hours ago
Base Salary
$150k - $180k/yr
Responsibilities
- Lead the data quality team in building systems that evaluate tasks across RL environments, synthetic data, benchmarks, and domain-specific workflows.
- Define data quality strategy through QC systems, standards, and experiments that grade agent outputs.
- Develop methods for validating synthetic data at scale, including failure-mode analysis, task mutation checks, and trajectory auditing.
- Partner with research engineers, domain experts, and data vendors to diagnose quality issues and improve data-generation workflows.
- Build internal tools, dashboards, validation pipelines, and feedback loops that turn research insights into production systems.
- Mentor research engineers and maintain high standards for technical rigor, clarity, and execution speed.
Requirements
- 5+ years of relevant engineering or research experience building systems for AI/ML data evaluation or data quality.
- Demonstrated experience leading technical teams through ambiguous projects from definition to implementation and iteration.
- Advanced proficiency in Python, Docker, and Linux environments.
- Experience with QC systems, evaluations, benchmarks, synthetic data pipelines, validation workflows, or model evaluation infrastructure.
- Research-oriented understanding of AI evaluations and post-training beyond surface-level agent tooling projects.
- Ability to design metrics, experiments, and QA/QC processes.
- Strong written communication and ability to explain methodology to diverse audiences.
- Experience working with subject-matter experts to convert domain judgment into scalable review or generation systems.
- Early-stage startup experience and ability to work independently in fast-paced environments.
Benefits
- Salary range is $150,000 to $180,000 USD annually.
- Visa sponsorship is available.
- On-site role in San Francisco, California, USA; the team also has a presence in Singapore.
Categories
Data EngineeringML Engineering