Robotics Data Pipeline Engineer – Multimodal Data

2 months ago

Houston, TX, USASenior

H1B Sponsor

Responsibilities

Architect end-to-end pipelines that ingest egocentric video, teleoperation sessions, and open datasets and produce indexed, queryable, training-ready datasets.
Create temporal segmentation, metadata and scene-graph extraction, embedding-based retrieval, and language-annotation workflows.
Design cross-modal validation for video, proprioception, force or haptic signals, language annotations, and robot state.
Orchestrate hand tracking, segmentation, depth estimation, 3D reconstruction, and pose tracking to retarget human demonstrations into robot trajectories.
Run simulation-in-the-loop validation for kinematic feasibility, physics replay, and motion consistency.
Implement spatial, temporal, viewpoint, and sensor-noise augmentation for expert trajectories and learning data.
Unify state-action representations across embodiments, coordinate frames, rotation conventions, gripper or hand parameterizations, and sampling rates.
Build dataset query, visualization, auditing, clip-browser, trajectory-viewer, and annotation-review tools and convert model-failure analyses into curation rules and recollection requests.

M.S. or Ph.D. in Computer Science, Data Engineering, Machine Learning, Robotics, Mechanical Engineering, or a related field.
Deep expertise in Python and extensive experience with PyTorch custom dataloaders for multimodal datasets.
Experience processing force-torque sensor, load-cell, or tactile-array time-series data and aligning it with visual frames.
Mastery of video-processing pipelines and libraries including OpenCV, FFmpeg, and Decord, with experience managing terabyte-scale video I/O bottlenecks.
Working knowledge of 3D geometry and robotics data, including coordinate frames, transforms, rotation representations, camera intrinsics and extrinsics, forward and inverse kinematics, and URDF.
Proven ability to implement programmatic and generative augmentation for computer vision and time-series data.
Bonus: experience with NVIDIA’s robotics software stack or datasets such as Open X-Embodiment, DROID, AgiBot World, and EgoDex.
Bonus: familiarity with segmentation, monocular depth, hand and body pose estimation, 6-DoF object pose tracking, and point tracking using tools such as SAM, MANO, and SMPL.
Bonus: familiarity with Ray and Apache Spark for distributed cluster computing.
Bonus: experience generating or using synthetic robotic data with Omniverse or MuJoCo.
Bonus: experience integrating spatial-awareness or tactile representations such as Fourier encoding into visual pipelines.

Competitive compensation and performance-based bonus (amounts not specified).
99% employer-covered medical benefits, early-stage equity, and competitive paid time off.
Company-wide paid winter break from December 24 through January 2.
Full-time role located in Houston, Texas or Pensacola, Florida.