2 days ago
Pune, India or Bengaluru, IndiaSenior
Responsibilities
- Design, develop, and maintain data solutions for data generation, collection, and processing.
- Build and optimize end-to-end ETL pipelines for migrating data from legacy on-premises systems to AWS.
- Use AWS Glue, PySpark, SQL scripts, Step Functions, EventBridge, and Lambda for data extraction, transformation, and loading.
- Handle high-volume datasets with error handling and retry mechanisms.
- Test pipelines for performance and deploy them with support from a DevOps engineer.
- Monitor and troubleshoot pipelines using CloudWatch and optimize workflows for cost and scalability.
- Support data quality validation and incremental loads to maintain data integrity.
- Collaborate with solution designers and multiple cross-functional teams and document code for team handover.
Requirements
- At least 5 years of experience with AWS Glue and PySpark.
- Proficiency in AWS Glue, PySpark, SQL, ETL, Unix, Iceberg, Astronomer, and data warehousing concepts.
- Strong understanding of data pipeline architecture and design principles.
- Experience with data warehousing solutions and ETL processes.
- Familiarity with cloud computing services and data storage solutions.
- Ability to troubleshoot and optimize data workflows for performance.
- Minimum 15 years of full-time education.
Benefits
- The position is based at the company's Pune office.
About Accenture
Accenture is a global professional services firm providing management consulting, systems integration and technology, cybersecurity, and business process outsourcing for enterprises and governments. It operates a services-driven model delivering projects and managed services, often with major cloud and software partners, across industries worldwide. Headquartered in Dublin and publicly traded on the NYSE (ACN), it originated as Andersen Consulting and adopted the Accenture name in 2001.