22 hours ago
Singapore, SingaporeMid Level / Senior
Base Salary
$100k - $170k/yr
Responsibilities
- Build and improve quality-control systems for training data created within reinforcement-learning environments.
- Define quality standards and design experiments and metrics to assess agent outputs.
- Develop auditing approaches, including sampling strategies and rule-based or model-assisted validation pipelines.
- Work with data vendors to diagnose quality issues, understand agent failure modes, and improve data-generation processes.
- Translate quality-control findings into improvements to internal infrastructure and vendor-facing tools.
Requirements
- Two to four years of experience in research engineering or a similar role, including hands-on quality-control automation.
- Proficiency with Python, Docker, and Linux, with experience building scalable data-validation and automated QA/QC systems end to end.
- Experience with benchmarks and evaluations for reinforcement-learning training data, including realistic tasks, reliable rubrics, and useful training trajectories.
- Ability to define and measure training-data quality and build quality-control systems grounded in human judgment rather than heavy reliance on large language models.
- Knowledge of statistics and experience designing metrics, experiments, and quality-assurance processes.
- Strong communication skills, curiosity, and the ability to work independently in ambiguous, fast-moving environments.
Benefits
- Annual salary range of $100,000 to $170,000 USD.
- Visa sponsorship is available.
- On-site work in Singapore.
- Fully remote independent-contractor arrangements may be considered for candidates based elsewhere, particularly in Europe.