Principal Architect - Reinforcement Learning (Physical AI)
Caterpillar2 days ago
Base Salary
$148k - $240k/yr
Responsibilities
- Define the reference architecture for reinforcement learning, imitation learning, and foundation-model post-training systems for physical AI.
- Design task, observation, action, reward, curriculum, policy, and model architectures for autonomous construction equipment.
- Architect large-scale closed-loop training using GPU-parallel simulators, hybrid or neural simulation, and fleet-scale data.
- Develop strategies for domain randomization, system identification, sim-to-real transfer, and real-to-sim reconstruction.
- Establish policy evaluation, regression, safety-case, benchmark, reproducibility, and deployment-gating standards.
- Lead the evaluation and deployment of new technologies and collaborate with cross-functional teams on digital solutions.
- Mentor ML engineers and researchers and turn research capabilities into repeatable, field-deployed engineering systems.
Requirements
- Typically 8+ years in robot learning with reinforcement learning through real-hardware deployment, or a PhD plus 5+ years of experience.
- Experience with PPO, SAC, GRPO, model-based or world-model RL, reward shaping, curriculum design, and closed-loop training.
- Experience with behavior cloning, offline RL including IQL and GAIL, distillation, diffusion policies, and RL-based foundation-model post-training.
- Experience architecting RL frameworks on GPU-parallel simulators and scaling training with Ray, Kubernetes, or Slurm.
- Experience with sim-to-real transfer, domain randomization, system identification, actuator and latency modeling, and field-task evaluation.
- Knowledge of learned-policy metrics, statistical policy comparison, continuous evaluation, regression testing, adversarial scenarios, runtime safeguards, and safety-case practices.
- Foundational knowledge of MPC, impedance control, trajectory optimization, perception-to-action pipelines, and real-time constraints.
- Experience shaping demonstration and fleet-data collection, auto-labeling, and synthetic-data strategies.
- Strong Python and PyTorch skills, working C++ experience, CUDA familiarity, experiment-tracking experience, and Bazel or monorepo tooling.
- Architecture leadership experience, including setting reproducibility and benchmark standards and shipping a learning-based capability.
- Off-highway, construction, mining, or robotics-autonomy experience is valuable; research or publication experience is a plus.
- Master’s or doctoral degree in Computer Science, Robotics, Machine Learning, Controls, or a related field preferred.
Benefits
- Medical, dental, and vision benefits, paid time off, 401(k) savings plans, HSA, FSAs, wellness programs, employee assistance, voluntary benefits, discounts, career development, incentive bonus, disability benefits, life insurance, parental and adoption benefits, and tuition reimbursement are available subject to eligibility.
- The role is full-time and onsite five days per week at the Irving, Texas office.
- Domestic relocation assistance is available.
- Visa sponsorship is available for eligible applicants.
Tech Stack
Categories
ML EngineeringRobotics
About Caterpillar
Caterpillar Inc. designs, manufactures, and services construction and mining equipment, diesel and natural gas engines, industrial gas turbines, and diesel-electric locomotives for infrastructure, resource, and energy customers. The public company, founded in 1925 and headquartered in Irving, Texas, sells primarily through an independent worldwide dealer network and provides equipment financing. It operates through Construction Industries, Resource Industries, and Energy & Transportation segments, with shares traded on the NYSE and Euronext Paris.