1 day ago
Base Salary
$180k - $300k/yr
Responsibilities
- Design benchmark tasks for LLMs, VLAs, and WAMs covering long-horizon planning, dexterous manipulation, and other robotics capabilities.
- Build verifiers, success detectors, progress metrics, and harnesses for evaluating models in simulation and on real hardware.
- Develop simulation pipelines and measure and improve the correlation between simulated and real-world performance.
- Fine-tune open-source VLAs consistently to establish fair evaluation baselines.
- Analyze failures across runs, identify capability gaps, and turn findings into targeted data collection and reinforcement-learning environments.
- Produce partner-facing evaluation results using rigorous statistics and clear reporting.
Requirements
- Strong programming skills in Python or C++.
- Experience building benchmarks, evaluation pipelines, or graders for machine-learning models and assessing what evaluation scores measure.
- Hands-on experience with robot learning, simulation such as MuJoCo or Isaac Sim, or deploying policies on real hardware.
- Experience working with LLMs or coding agents and diagnosing model failures from execution traces.
- Experience building and shipping software in fast-moving environments.
- Bonus: contributions to robotics or agent benchmarks such as LIBERO, BEHAVIOR, RoboArena, or Terminal-Bench.
- Bonus: experience with reinforcement learning, reward modeling, post-training, real2sim, system identification, sim-to-real transfer, or evaluation statistics.
- Bonus: familiarity with containerized evaluation frameworks such as Harbor.
About XDOF
XDOF builds data, software, and operational infrastructure for robotics companies and research labs developing general‑purpose robots and physical AI. It provides scalable data collection, tooling, hardware integration, and services that help teams train, deploy, and manage autonomy systems. The company is privately held and positions itself as an infrastructure partner to frontier labs and robotics builders, with a team drawing experience from Tesla, Google, Applied Intuition, BAIR, Covariant, and Gatik.