Staff, Robotics ML/Data Engineer

5 hours ago

Houston, TX, USAStaff+

Responsibilities

Architect end-to-end multimodal data ingestion pipelines that convert egocentric video, teleoperation recordings, and third-party datasets into indexed, queryable, training-ready datasets.
Design cross-modal validation systems for video, proprioception, force/haptic signals, language annotations, and robot state.
Orchestrate hand tracking, segmentation, depth estimation, 3D reconstruction, pose tracking, human-to-robot trajectory retargeting, and simulation-in-the-loop validation.
Implement spatial, temporal, viewpoint, and sensor-noise augmentation for expert trajectories and multimodal learning data.
Create unified state-action representations across embodiments, coordinate frames, rotation conventions, gripper or hand parameterizations, and sampling rates.
Build tools for dataset querying, visualization, auditing, clip browsing, trajectory review, and annotation workflows.
Perform temporal segmentation, metadata and scene-graph extraction, embedding-based retrieval, and language annotation of recordings.
Use model-failure analyses to develop data curation rules and targeted recollection requests.

M.S. or Ph.D. in Computer Science, Data Engineering, Machine Learning, Robotics, or a related field.
Deep expertise in Python and extensive experience with PyTorch, including custom dataloaders for multimodal datasets.
Experience processing force-torque sensor, load-cell, or tactile-array time-series data and aligning it with visual frames.
Mastery of video-processing pipelines and OpenCV, FFmpeg, and Decord for terabyte-scale video datasets.
Working knowledge of 3D geometry, coordinate frames and transforms, rotation representations, camera intrinsics and extrinsics, forward and inverse kinematics, and URDF.
Proven ability to implement programmatic and generative data augmentation for computer vision and time-series data.
Bonus: experience with Open X-Embodiment, DROID, AgiBot World, EgoDex, or similar robotics datasets or software.
Bonus: familiarity with SAM-family segmentation, monocular depth, MANO or SMPL hand/body pose estimation, 6-DoF object pose tracking, and point tracking.
Bonus: familiarity with Ray or Apache Spark for distributed data processing.
Bonus: experience generating or using synthetic robotic data with Omniverse or MuJoCo.
Bonus: experience integrating spatial-awareness or tactile representations such as Fourier encoding into visual pipelines.