1 day ago
Remote, United StatesSenior
Base Salary
$180k - $250k/yr
Responsibilities
- Design benchmarks and evaluation suites for manufacturing and supply chain workflows.
- Work with domain experts to define tasks, reference answers, rubrics, and graders.
- Train and fine-tune models using evaluation results to guide experiments.
- Identify agent failures, develop improvements, and validate them with measured results.
- Help establish research and evaluation practices as an early team member.
Requirements
- At least 3 years of experience with senior- or staff-level capability.
- Experience designing and writing benchmarks or evaluations.
- Experience authoring tasks, defining scope, creating reference solutions or rubrics, and checking task quality.
- Hands-on experience training and fine-tuning models beyond using models through APIs.
- Ability to collaborate with non-ML domain experts and translate their knowledge into measurable tasks.
- Supply chain, procurement, manufacturing, or early-stage startup experience is a plus.
Benefits
- Fully remote within the United States.
- Equity included.
Categories
AI ResearchML Engineering
