5 months ago
Base Salary
$130k - $500k/yr
Responsibilities
- Build ML systems that score, validate, and improve complex work products with imperfect labels.
- Design evaluation frameworks for ambiguous tasks with partial, delayed, or disputed ground truth.
- Turn review, disagreement, correction, and adjudication into measurable model and system improvements.
- Own production ML quality, regression detection, drift, latency, cost, and explainability.
- Improve model behavior through prompting, fine-tuning, retrieval, active learning, heuristics, and error analysis.
- Integrate inference into durable, long-running workflows with backend engineers while preserving debuggability and human oversight.
Requirements
- Track record of shipping ML systems that improved a real product, workflow, or business metric.
- Strong judgment in model quality, evaluation design, error analysis, and production failure modes.
- Comfort working in ambiguous problem spaces with imperfect labels and evolving definitions of correctness.
- Ability to choose appropriately among prompting, fine-tuning, heuristics, retrieval, human review, and simpler product constraints.
- Solid engineering fundamentals across the full ML stack, beyond modeling alone.
- Familiarity with LLM applications, model-assisted workflows, evaluation frameworks, or human-in-the-loop ML is preferred.
Benefits
- In-person work five days per week from the San Francisco, New York City, or London office.
- Bi-annual performance bonus structure and equity grant.
- Relocation and housing support may be available.
- Meal, laundry, and personal wellness reimbursement programs.
- Free Equinox membership.
- Health, dental, and vision insurance.
Tech Stack
Categories
About Mercor
Mercor builds an expert-powered platform that trains, evaluates, and deploys AI systems for AI labs and enterprises. It operates APEX to assess model performance on economically valuable tasks, and provides services such as custom AI agents and teams of vetted domain experts who encode organizational knowledge into AI. Founded in 2023 and headquartered in San Francisco, it is privately held and works with frontier AI labs and enterprise clients requiring strict data isolation.