2 months ago
Base Salary
$250k - $280k/yr
Responsibilities
- Build RL environments for agentic tasks, including task definitions, tool surfaces, state and reset semantics, reward design, and parallel execution harnesses
- Develop programmatic checks, LLM judges, rubric pipelines, and pass@k scoring systems
- Build SFT and RL fine-tuning pipelines from data collection through training and checkpoint evaluation
- Develop evaluation systems that run millions of agent trajectories
- Build scalable training and serving infrastructure with orchestration, fault tolerance, and cost accounting
- Create tooling, CI, harnesses, and libraries that support reliable engineering throughput
- Document findings from experiments so they inform future team decisions
Requirements
- At least 3 years of experience shipping systems relied on by customers or engineers
- Deep proficiency in Python and comfort across the rest of the stack
- Strong system and API design judgment with the ability to make and defend architecture decisions
- Experience fine-tuning models for agentic tasks using SFT and at least one RL method such as GRPO, PPO, or DPO in production
- Experience building agent environments and designing verifiers or graders for open-ended work
- Ability to debug training runs methodically and reason about experiment compute economics
- Preferred experience with agent harnesses, coding agents, sandboxing, egress control, credential handling, distributed systems, ML infrastructure, data systems at scale, or frontier labs
Benefits
- Hybrid work in San Francisco with 3 days per week in the office
- Career advancement opportunities tied to impact
- Fast-paced, high-ownership work environment
Tech Stack
Categories
About Labelbox
Labelbox builds a data platform and managed services for creating, managing, and evaluating training data for AI models, including computer vision, NLP, and RLHF workflows. The company sells subscriptions and labeling services to enterprises and AI labs; founded in 2018 and headquartered in San Francisco, it is privately held and used by Fortune 500 customers.