Lead Data Engineer – Physical AI Platform, Data Engineering
Caterpillar1 day ago
Base Salary
$128k - $209k/yr
Responsibilities
- Collaborate with Principal Software Engineers and Data Architects to define solution architecture.
- Lead the design and optimization of scalable Python data pipelines and microservices for real-time and batch processing.
- Develop cloud-native data ingestion and streaming solutions using AWS services including Kinesis, S3, DynamoDB, and EventBridge.
- Own data integration frameworks and source data pipelines supporting CI Autonomy initiatives.
- Translate stakeholder requirements into scalable data architectures, workflows, mappings, and system designs.
- Establish automated testing, data quality controls, and validation frameworks across distributed data ecosystems.
- Lead monitoring, performance tuning, and root-cause analysis of production data platforms using observability tools such as CloudWatch.
Requirements
- Bachelor’s degree in Computer Science, Computer Engineering, or a related field.
- 8+ years of experience in data engineering or related disciplines with increasing responsibility.
- Extensive experience with modern, large-scale, complex Caterpillar data platforms such as Helios Data Platform.
- Strong production experience developing and deploying Python solutions.
- Experience leading teams building high-throughput, scalable data pipelines.
- Strong hands-on experience with AWS data services at scale, including Kinesis, S3, DynamoDB, and EventBridge.
- Strong SQL skills, including data quality and validation practices.
- Experience deploying software using Azure DevOps, Jira, Jenkins, or similar CI/CD tools.
- Experience developing microservices for real-time data ingestion and applications using relational and NoSQL databases.
- Ability to maintain data integrity across distributed and streaming systems and experience with monitoring, testing, and automation in large-scale data environments.
Benefits
- Medical, dental, and vision benefits; paid time off; 401(k) savings plans; HSA and FSA options; wellness programs; employee assistance; voluntary benefits and discounts; career development; incentive bonus; disability and life insurance; parental and adoption benefits; and tuition reimbursement.
- Full-time position based in the Chicago, Illinois office and requiring onsite work five days per week.
- Domestic relocation assistance is available.
- Visa sponsorship is available for eligible applicants.
About Caterpillar
Caterpillar Inc. designs, manufactures, and services construction and mining equipment, diesel and natural gas engines, industrial gas turbines, and diesel-electric locomotives for infrastructure, resource, and energy customers. The public company, founded in 1925 and headquartered in Irving, Texas, sells primarily through an independent worldwide dealer network and provides equipment financing. It operates through Construction Industries, Resource Industries, and Energy & Transportation segments, with shares traded on the NYSE and Euronext Paris.