23 hours ago
Kolkāta, India +4 moreMid Level
Responsibilities
- Assist with data platform blueprint and component design in collaboration with Integration Architects and Data Architects.
- Build, maintain, and optimize scalable batch and real-time data pipelines for large-scale processing.
- Develop ETL/ELT processes using PySpark, Spark, and related tools to extract, transform, and load data.
- Design and deploy data engineering solutions on AWS, Azure, or GCP.
- Support Big Data processing frameworks and data lakes for structured and unstructured data.
- Implement secure and efficient data ingestion across different storage solutions.
- Develop reusable data pipeline code and tune Spark jobs for scalability and performance.
- Collaborate with data scientists, analysts, and engineers to ensure data accuracy and availability.
- Support data governance and security practices across cloud platforms.
Requirements
- At least 3 years of professional experience is required.
- 15 years of full-time education is required.
- Must have experience with the Databricks Unified Data Analytics Platform.
- Strong proficiency with PySpark, Spark, or similar scalable data-processing frameworks.
- Experience with AWS, Azure, or GCP for data engineering workflows.
- Knowledge of Big Data architectures, data storage, data processing, and cloud-native storage such as S3, Blob Storage, BigQuery, or Redshift.
- Experience with Apache Airflow or similar data orchestration tools.
- Knowledge of Parquet, Avro, or JSON data formats.
- Strong Python coding skills for data pipeline development and good understanding of SQL and database technologies.
- Strong troubleshooting, debugging, and performance optimization skills.
- Preferred experience with real-time processing, Kafka, dbt, Informatica, Talend, or Matillion, plus exposure to CI/CD and DevOps practices.
Tech Stack
Amazon RedshiftApache AirflowApache KafkaApache SparkAWSAzureDatabricksdbtGoogle BigQueryGoogle Cloud PlatformInformaticaPythonSQLTalend
Categories
Data Engineering
About Accenture
Accenture is a global professional services firm providing management consulting, systems integration and technology, cybersecurity, and business process outsourcing for enterprises and governments. It operates a services-driven model delivering projects and managed services, often with major cloud and software partners, across industries worldwide. Headquartered in Dublin and publicly traded on the NYSE (ACN), it originated as Andersen Consulting and adopted the Accenture name in 2001.