Evaluation Engineer
Maven Robotics3 days ago
Responsibilities
- Own the architecture, implementation, and evolution of the robot learning evaluation framework across simulation and real robots.
- Define task specifications, success criteria, failure taxonomies, evaluation metrics, and benchmark protocols for manipulation policies.
- Build automated systems for model comparison, regression detection, large-scale evaluations, and performance reporting.
- Develop simulation environments, task scenarios, and procedural variations for policy evaluation.
- Build reproducible real-robot evaluation workflows that minimize manual supervision and produce statistically meaningful results.
- Measure and improve the relationship between simulated and real-world policy performance.
- Build tools for replaying, visualizing, and diagnosing policy failures across simulated and physical robots.
- Collaborate with robot learning researchers and engineers across simulation, perception, controls, data, and infrastructure to identify failure sources.
Requirements
- Significant experience building robotics evaluation, simulation, testing, or benchmarking systems.
- Strong programming ability in Python, C++, or similar robotics or systems languages.
- Experience with robotics simulation platforms such as Isaac Sim, MuJoCo, Gazebo, or similar systems.
- Experience evaluating learned robotic policies in simulation and on physical robots.
- Experience designing reproducible experiments, metrics, benchmarks, and failure-analysis workflows.
- Ability to build reliable software for automated testing, data collection, visualization, and reporting.
- Strong understanding of robotics systems and the ability to debug across software, models, sensors, controls, and hardware.
- Track record of independently owning complex technical projects from initial design through deployment and ongoing operation.
- Familiarity with sim-to-real transfer, domain randomization, sensor modeling, and system identification is a plus.
- Experience configuring simulation assets, hardware-in-the-loop testing, continuous integration for robotics, automated robot test cells, learned manipulation architectures, or statistical analysis is a plus.
About Maven Robotics
Maven Robotics is building the world’s leading general-purpose AI robots.