5 hours ago
Singapore (city not specified)Mid Level
Base Salary
$100k - $170k/yr
Responsibilities
- Automate quality control for training data generated through the platform.
- Define and enforce standards for training-data quality.
- Design experiments and metrics to evaluate agent outputs.
- Build scalable data-validation pipelines and automated QA/QC systems.
- Investigate quality issues and diagnose agent failure modes with data vendors.
- Develop sampling strategies and rule-based or model-assisted validation pipelines.
- Incorporate quality insights into infrastructure tools and vendor workflows to reduce anomalies and edge cases.
Requirements
- Two to four years of experience in research engineering or a related technical role, especially quality-control automation.
- A technical degree and proficiency with Python, Docker, and Linux.
- Experience independently building scalable data-validation pipelines and end-to-end automated QA or QC systems.
- Experience with reinforcement-learning data benchmarks and evaluations, including task design, reliable rubrics, and useful training trajectories.
- Ability to define and measure training-data quality and design systems using human judgment without heavy reliance on large language models.
- Experience designing evaluation metrics and experiments and collaborating with data vendors to resolve quality problems.
- Knowledge of statistics, strong communication skills, curiosity, and the ability to work independently in an early-stage environment.
Benefits
- Visa sponsorship is available.
- The role is on-site in Singapore, Singapore.