1 month ago
Thiruvananthapuram, India +4 moreStaff+
Responsibilities
- Assist with data platform blueprints and designs in collaboration with Integration Architects and Data Architects.
- Lead the design, development, and optimization of scalable data pipelines using PySpark, Spark, and big data technologies.
- Architect cloud-based data engineering solutions across AWS, Azure, or GCP.
- Design and build reusable ETL/ELT pipelines for data lakes and data warehouses.
- Develop and maintain large-scale batch, real-time, and analytics data processing systems.
- Optimize Spark and big data processing jobs for high-volume workloads.
- Enforce data security, privacy, compliance, and governance practices across cloud platforms.
- Implement automated data testing and monitoring for data quality and pipeline reliability.
- Lead cross-functional teams implementing cloud-based data platforms and workflows.
- Guide and mentor junior engineers and advocate for continuous improvement in data engineering.
Requirements
- At least 12 years of experience is required.
- Expert-level knowledge of PySpark, Spark, and big data technologies.
- Proven experience designing data engineering workflows on AWS, Azure, or GCP.
- Extensive experience with data pipeline architecture, design, optimization, and large-scale ETL/ELT processes.
- Hands-on experience integrating data into S3, Blob Storage, BigQuery, and Redshift.
- Proficiency in Python, SQL, and scripting for data pipeline development.
- Experience with Kafka for real-time data streaming and batch processing tools.
- Experience leading cross-functional teams implementing cloud-based data platforms.
- Understanding of data security, compliance, and governance in cloud environments.
- Familiarity with dbt is preferred.
- Exposure to CI/CD, DevOps practices, Docker, and Kubernetes is preferred.
- Certifications or experience with Apache Airflow, Fivetran, Informatica, or Talend are preferred.
- Experience with Power BI, Tableau, or Looker is preferred.
- Completion of 15 years of full-time education is required.
Tech Stack
Amazon RedshiftApache AirflowApache KafkaApache SparkAWSAzureDatabricksdbtDockerGoogle BigQueryGoogle Cloud PlatformInformaticaKubernetesPythonSQLTalend
Categories
Data Engineering
About Accenture
Accenture is a global professional services firm providing management consulting, systems integration and technology, cybersecurity, and business process outsourcing for enterprises and governments. It operates a services-driven model delivering projects and managed services, often with major cloud and software partners, across industries worldwide. Headquartered in Dublin and publicly traded on the NYSE (ACN), it originated as Andersen Consulting and adopted the Accenture name in 2001.