13 hours ago
Bengaluru, IndiaSenior
Responsibilities
- Develop scalable and reliable ETL/ELT data pipelines using Python, AWS Glue, Lambda, and Step Functions.
- Design and optimize S3-based data lakes, including partitioning, lifecycle policies, storage classes, and Glue Data Catalog metadata.
- Implement IAM policies, S3 access controls, KMS encryption, and CloudTrail/CloudWatch audit logging.
- Package and deploy data jobs and functions, maintain unit and integration tests, and support CI/CD pipelines.
- Build event-driven processing with S3 events and Lambda triggers while optimizing cost, concurrency, and resilience.
- Implement data validation, schema enforcement, logging, monitoring, alerting, and pipeline observability.
- Collaborate with data platform, analytics, and governance teams to ensure reliable, scalable, and compliant solutions.
- Where applicable, develop Databricks notebooks and jobs and optimize Delta Lake using Z-ordering, OPTIMIZE, and VACUUM.
Requirements
- At least 5 years of experience with AWS Glue, Python, and PySpark.
- Strong understanding of data integration and ETL processes.
- Experience with cloud computing services and architectures.
- Familiarity with software development lifecycle practices and agile methodologies.
- 15 years of full-time education is required.
Benefits
- The position is based in the Accenture Bengaluru office.
About Accenture
Accenture is a global professional services firm providing management consulting, systems integration and technology, cybersecurity, and business process outsourcing for enterprises and governments. It operates a services-driven model delivering projects and managed services, often with major cloud and software partners, across industries worldwide. Headquartered in Dublin and publicly traded on the NYSE (ACN), it originated as Andersen Consulting and adopted the Accenture name in 2001.