4 hours ago
Singapore, SingaporeMid Level / Senior
Base Salary
$150k - $250k/yr
Responsibilities
- Design and build automated quality-control systems for training data using human judgment rather than heavy LLM reliance.
- Define and enforce quality standards for reinforcement-learning training data across tasks and environments.
- Design experiments and metrics to evaluate and grade agent outputs.
- Partner with data vendors to diagnose quality issues, agent failure modes, and data-generation workflow problems.
- Build auditing systems with sampling strategies and rule-based or model-assisted validation pipelines.
- Feed QC findings back into infrastructure tooling and the vendor portal to reduce anomalies, inconsistencies, and edge cases.
Requirements
- 2–4 years of experience in engineering or research roles focused on QC automation or data quality.
- Strong proficiency in Python, Docker, and Linux environments.
- Experience building scalable data-validation pipelines and automated QA/QC systems end-to-end.
- Experience with benchmarks and evaluations for reinforcement-learning training data, including realistic tasks, reliable rubrics, and useful trajectories.
- Ability to define and measure training-data quality using human understanding rather than delegating judgment to LLMs.
- Experience designing metrics and experiments to grade agent outputs.
- Comfort applying statistics to QA/QC process design.
- Strong written and verbal communication skills, curiosity, autonomy, and ability to work in fast-paced, unstructured startup environments.
Benefits
- Visa sponsorship is available.
- The role is on-site in Singapore.
- Fully remote independent-contractor arrangements may be considered for candidates outside Singapore, particularly in Europe.