Software Engineer - Data Engineering
Caterpillar15 hours ago
Chennai, India or Bengaluru, IndiaSenior
Responsibilities
- Design, implement, deploy, and maintain end-to-end near real-time data pipelines and data movement solutions.
- Build and support large-scale data lakes, data warehouses, and robust data platforms for OLTP and OLAP workloads.
- Develop software and infrastructure components independently, including complex systems used by multiple applications or systems.
- Apply software-quality practices, integrated testing, customer acceptance testing, test-driven development, and behavior-driven development.
- Communicate with end users and internal customers to direct development, debugging, testing, and quality improvements.
- Mentor junior developers and identify opportunities for team growth and technical excellence.
- Demonstrate leadership on medium- to large-scale projects affecting strategic priorities.
- Coordinate work, set priorities, and resolve complex technical problems with minimal management input.
Requirements
- Bachelor’s degree in computer science, electrical engineering, or a related field from an accredited college or university.
- At least 5 years of software development experience, or at least 3 years with a master’s degree in computer science or a related field.
- At least 5 years of experience designing and developing data pipelines in Python.
- Experience designing, developing, deploying, and maintaining software at scale, including concurrency concepts.
- Strong Python object-oriented programming experience and knowledge of computer science fundamentals, data structures, algorithms, and object-oriented design.
- In-depth understanding of big-data pipelines and expertise building large data lakes, data warehouses, and end-to-end near real-time pipelines.
- Expertise with SQL and NoSQL databases, message brokers, AWS or Azure cloud services, and software deployment tooling.
- At least 2 years of experience deploying and maintaining software using public clouds such as AWS or Azure.
- Strong analytical skills, leadership on medium- to large-scale projects, and the ability to work under pressure and within time constraints.
- Understanding of AI concepts and current developments, including Gen AI, MCP, and ATA.
Benefits
- The position requires working onsite five days per week.
- Relocation is available for this position.
Tech Stack
Categories
Data Engineering
About Caterpillar
Caterpillar Inc. designs, manufactures, and services construction and mining equipment, diesel and natural gas engines, industrial gas turbines, and diesel-electric locomotives for infrastructure, resource, and energy customers. The public company, founded in 1925 and headquartered in Irving, Texas, sells primarily through an independent worldwide dealer network and provides equipment financing. It operates through Construction Industries, Resource Industries, and Energy & Transportation segments, with shares traded on the NYSE and Euronext Paris.