12 hours ago
Singapore, SingaporeStaff+
Base Salary
$150k - $180k/yr
Responsibilities
- Lead the data quality team in building systems that evaluate tasks across reinforcement-learning environments, synthetic data, benchmarks, and domain-specific workflows.
- Define data quality strategy through QC systems, standards, and experiments for grading agent outputs.
- Develop scalable synthetic-data validation methods, including failure-mode analysis, task mutation checks, and trajectory auditing.
- Partner with research engineers, domain experts, and data vendors to diagnose quality issues and improve data-generation workflows.
- Build internal tools, dashboards, validation pipelines, and feedback loops from qualitative research insights.
- Mentor research engineers and maintain high standards for technical rigor, clarity, and execution speed.
Requirements
- 5+ years of engineering or research experience building AI/ML data evaluation or data-quality systems.
- Demonstrated experience leading technical projects or teams through definition, implementation, and iteration on ambiguous problems.
- Advanced proficiency in Python, Docker, and Linux environments.
- Background as an ML researcher or engineer focused on evaluating AI training data at scale rather than traditional data science or analytics.
- Experience designing metrics, experiments, and QA/QC processes.
- Experience collaborating with subject-matter experts, vendors, and domain specialists to convert judgment into scalable review or generation systems.
- Research-oriented understanding of AI evaluations and post-training pipelines.
- Strong written communication skills and the ability to explain methodology to mixed audiences.
- Early-stage startup experience, independent working ability, and strong attention to subtle inconsistencies and edge cases.
Benefits
- Salary range of $150,000 to $180,000 USD annually.
- Visa sponsorship is available.
- On-site work in Singapore.