3 days ago
Singapore, SingaporeMid Level / Senior
Base Salary
$150k - $250k/yr
Responsibilities
- Automate quality control for training data produced through the platform’s infrastructure.
- Define and enforce quality standards for reinforcement-learning training data.
- Design experiments, metrics, benchmarks, rubrics, and evaluations to grade agent outputs.
- Build auditing systems with sampling strategies and rule-based or model-assisted validation pipelines.
- Partner with data vendors to debug quality issues, diagnose agent failure modes, and improve data-generation processes.
- Feed QC findings back into infrastructure tools and the data vendor portal to reduce anomalies, inconsistencies, and edge cases.
Requirements
- 2 to 4 years of experience in an engineering or research role focused on QC automation or a closely related area.
- Proficiency in Python, Docker, and Linux environments.
- Experience building scalable data-validation pipelines and automated QA/QC systems end-to-end.
- Experience with benchmarks and evaluations for reinforcement-learning training data, including realistic tasks, reliable rubrics, and useful trajectories.
- Experience defining and measuring training-data quality standards and designing experiments to grade agent outputs.
- Hands-on experience partnering with data vendors to debug quality issues and provide actionable feedback.
- Solid knowledge of statistics and comfort designing metrics and QA/QC processes.
- Strong written and verbal communication skills for cross-time-zone collaboration.
- Ability to work autonomously in an unstructured, fast-paced, early-stage startup environment.
Benefits
- Salary range of $150,000 to $250,000 USD annually.
- Visa sponsorship is available.
- On-site in Singapore.
- Fully remote independent-contractor arrangements may be considered for candidates outside Singapore, particularly in Europe.