1 day ago
Chennai, India or Bengaluru, IndiaSenior
Responsibilities
- Design application architectures for custom software systems that meet business needs and provide scalable, secure, high-performing solutions.
- Build, maintain, and optimize scalable data pipelines for large-scale processing.
- Develop ETL/ELT processes using PySpark, Spark, and related tools.
- Design and deploy data solutions on AWS, Azure, or GCP.
- Support Big Data processing frameworks and data lakes for structured and unstructured data.
- Implement secure and efficient data ingestion across storage solutions and support batch and real-time processing.
- Collaborate with data scientists, analysts, and engineers to ensure data accuracy and availability.
- Tune Spark jobs and data pipelines for scalability and efficiency.
- Maintain reusable ETL code and support data governance and security practices.
Requirements
- At least 5 years of experience.
- Required experience with Databricks Unified Data Analytics Platform.
- 15 years of full-time education.
- Strong proficiency with PySpark, Spark, or similar scalable data-processing frameworks.
- Experience with AWS, Azure, or GCP data engineering workflows.
- Understanding of Big Data architecture, storage, and processing concepts.
- Familiarity with S3, Blob Storage, BigQuery, or Redshift.
- Experience with Apache Airflow or similar orchestration tools.
- Knowledge of Parquet, Avro, or JSON data formats.
- Strong Python coding skills and good understanding of SQL and database technologies.
- Excellent troubleshooting, debugging, and performance optimization skills.
- Experience with real-time processing and Kafka or similar tools is preferred.
- Exposure to cloud CI/CD and DevOps practices, dbt, Informatica, Talend, or Matillion is preferred.
- Cloud Data Architecture is a preferred skill.
Tech Stack
Amazon RedshiftApache AirflowApache KafkaApache SparkAWSAzureDatabricksdbtGoogle BigQueryGoogle Cloud PlatformInformaticaPythonSQLTalend
Categories
Data Engineering
About Accenture
Accenture is a global professional services firm providing management consulting, systems integration and technology, cybersecurity, and business process outsourcing for enterprises and governments. It operates a services-driven model delivering projects and managed services, often with major cloud and software partners, across industries worldwide. Headquartered in Dublin and publicly traded on the NYSE (ACN), it originated as Andersen Consulting and adopted the Accenture name in 2001.