Technical Lead - Autonomy Evaluation
CloudKitchens3 days ago
Base Salary
$185k - $242k/yr
Responsibilities
- Own safety and behavior metrics, including collision, near-miss, trajectory-agreement, comfort, and release go/no-go criteria.
- Build and operate log-replay and scoring pipelines for open-loop and closed-loop evaluation of perception, localization, and planning.
- Own regression benchmarking, evaluation dataset design, regression detection, and triage across candidate software releases.
- Mine evaluation corpora for rare events and improve scenario-class coverage using sampling and expected-value strategies.
- Ensure deterministic, reproducible evaluation execution with traceability to software versions and tracked replay cost.
- Set practices for metric definitions, test design, code review, and evaluation-result documentation.
- Mentor engineers, establish technical standards, and participate in hiring.
Requirements
- Bachelor's degree in Computer Science, Robotics, Electrical Engineering, Statistics, or a related field.
- 8+ years of relevant experience.
- 2+ years as a technical lead or engineering manager for an evaluation, validation, or test team.
- Experience owning an offline or system-level evaluation system for an autonomous vehicle, robotics program, or comparable production-scale ML model evaluation system.
- Experience designing and defending safety or behavior metrics for autonomous systems.
- Hands-on experience building log replay at scale and scoring against ground truth in open-loop and closed-loop evaluation.
- Experience running recurring evaluation pipelines on GPU clusters or comparable large-scale compute with responsibility for reproducibility, throughput, and cost.
- Experience with evaluation dataset design, benchmarking, and regression detection across software or model versions.
- Working command of applied statistics for system validation, including uncertainty, sampling, rare-event analysis, and data-volume requirements.
- Understanding of autonomous systems end to end, including sensing, perception, localization, planning, and controls.
- Experience with an autonomous vehicle or robotics platform and with robotics middleware and logging and replay tooling.
- Experience handling camera, LiDAR, and radar data across multiple modalities and timestamps.
- Proficiency in Python and ability to read, navigate, and debug existing C++ codebases.
Benefits
- Medical, dental, vision, disability, and life insurance.
- Flexible Spending Account and Health Savings Account options.
- 401(k), equity eligibility, sick time, unlimited flexible time off, paid holidays, paid parental leave, and pre-tax commuter benefits.
- Team lunch in the SoMa office every Tuesday and Thursday.
- The role is based onsite in San Francisco and office-based teams work onsite five days per week.
About CloudKitchens
CloudKitchens builds delivery-first commercial kitchen facilities and restaurant software for food and beverage operators. It leases turnkey kitchen space and provides tools for order management, delivery logistics, and multi-brand operations, enabling restaurants and entrepreneurs to expand with lower upfront costs. The company is privately held and headquartered in Los Angeles, serving markets across the U.S. with a mix of owned sites and enterprise solutions.