Senior Data Quality Assurance Engineer, Data Annotation Services
Caterpillar19 hours ago
Base Salary
$113k - $183k/yr
Responsibilities
- Define and maintain annotation guidelines, acceptance criteria, and quality standards for perception and VLA datasets.
- Design QA strategies for human, automated, and AI-assisted labeling and captioning, including sampling, audits, and golden-set validation.
- Build automated validation and test frameworks for labeling accuracy, consistency, completeness, schema conformance, and camera/LiDAR synchronization.
- Develop quality metrics, dashboards, and alerts covering precision, recall, IoU, inter-annotator agreement, defect rates, and dataset health.
- Identify and root-cause labeling defects while partnering with annotation providers, ML engineers, and domain experts on improvements.
- Build quality gates into data pipelines supporting labeling, human review, validation, and dataset publication.
- Evaluate auto-labeling and auto-captioning platforms including NVIDIA Cosmos Curator and Cosmos Reason.
- Integrate QA workflows with Caterpillar data systems, external annotation providers, and human-in-the-loop processes.
- Establish quality standards for AI training data supporting computer vision, robotics, autonomy, and multimodal AI.
Requirements
- Bachelor’s degree in Computer Science, Software Engineering, Data Science, Artificial Intelligence, Machine Learning, Robotics, or a related technical field.
- Production-grade Python experience for data validation, test automation, and data processing.
- Experience with version control, code review, CI/CD, Docker, and Kubernetes.
- Experience designing automated data quality checks and testing frameworks for data pipelines, APIs, and datasets using tools such as Great Expectations or pytest.
- Working knowledge of machine learning evaluation metrics including precision, recall, F1 score, IoU, mAP, and error analysis.
- At least 2 years of experience with camera and LiDAR sensor datasets for perception or machine learning applications.
- At least 3 years of experience in data annotation and quality assurance for autonomous systems or computer vision projects.
- Experience with annotation and dataset management platforms such as Labelbox, Supervisely, and RoboFlow.
- Experience with SQL, pandas, and Jupyter notebooks for dataset profiling, defect investigation, and quality reporting.
- Preferred experience with computer vision, robotics, autonomy, Physical AI, or multimodal AI systems.
- Preferred experience validating AI-assisted annotation, auto-labeling, and machine learning data pipelines.
- Preferred experience with PyTorch, TensorFlow, ONNX, Weights & Biases, or MLflow.
- Preferred experience with NVIDIA Cosmos Curator, Cosmos Reason, or similar AI automation platforms.
- Preferred experience with VLA datasets, caption quality evaluation, radar and autonomous machine sensor data, cloud platforms, data observability, MLOps, data pipeline orchestration, and human-in-the-loop quality workflows.
Benefits
- Medical, dental, and vision benefits.
- Paid time off including vacation, holidays, and volunteer time.
- 401(k) savings plans, Health Savings Account, and Flexible Spending Accounts.
- Health Lifestyle Programs, Employee Assistance Program, voluntary benefits, and employee discounts.
- Career development, tuition reimbursement, incentive bonus, disability benefits, life insurance, parental leave, and adoption benefits.
- Full-time onsite work five days per week at the Irving, Texas office.
- Domestic relocation assistance is available.
- Visa sponsorship is available for eligible applicants.
Tech Stack
Categories
About Caterpillar
Caterpillar Inc. designs, manufactures, and services construction and mining equipment, diesel and natural gas engines, industrial gas turbines, and diesel-electric locomotives for infrastructure, resource, and energy customers. The public company, founded in 1925 and headquartered in Irving, Texas, sells primarily through an independent worldwide dealer network and provides equipment financing. It operates through Construction Industries, Resource Industries, and Energy & Transportation segments, with shares traded on the NYSE and Euronext Paris.