1 month ago
Kochi, IndiaSenior / Staff+
Responsibilities
- Design, develop, and maintain scalable batch and real-time data pipelines using GCP, BigQuery, Python, SQL, Dataflow, Pub/Sub, Dataproc, Cloud Storage, and Cloud Composer.
- Build ETL/ELT ingestion, transformation, and integration workflows for structured, semi-structured, and unstructured data.
- Design and maintain BigQuery datasets, tables, views, materialized views, partitioning, clustering, and analytical data models.
- Optimize SQL queries, data processing workloads, storage structures, pipeline architecture, performance, and cloud consumption costs.
- Build curated data layers and reusable datasets supporting reporting, BI, analytics, machine learning, and downstream applications.
- Implement data validation, reconciliation, data quality checks, error handling, lineage, monitoring, alerting, and operational controls.
- Orchestrate pipelines using Cloud Composer, Apache Airflow, Workflows, or equivalent technologies.
- Collaborate with architects, data scientists, analysts, application teams, and business stakeholders to translate requirements into scalable technical solutions.
- Provide technical guidance, conduct code reviews, establish development standards, and support junior engineers.
- Promote engineering practices involving Git-based development, automated testing, CI/CD, Infrastructure as Code, documentation, monitoring, security, and production support.
Requirements
- 5–8 years of experience in data engineering, data warehousing, ETL/ELT development, SQL development, or large-scale data processing.
- Graduation-level educational qualification.
- Strong hands-on experience designing and implementing production data solutions on Google Cloud Platform.
- Strong production experience with BigQuery, including SQL development, partitioning, clustering, materialized views, query optimization, execution analysis, and cost management.
- Strong Python programming experience for data processing, pipeline development, automation, and integration.
- Advanced SQL skills including complex joins, aggregations, CTEs, window functions, analytical queries, optimization, and troubleshooting.
- Experience designing and supporting scalable batch and streaming data pipelines and large-scale cloud data solutions.
- Understanding of batch processing, streaming architecture, ETL/ELT patterns, data lakes, data warehouses, cloud data platforms, and analytical data modeling.
- Preferred experience with Dataform, dbt, or similar SQL-based transformation frameworks.
- Advantageous experience with Cloud Functions, Cloud Run, Workflows, or other serverless GCP services.
- Understanding of GCP security concepts including IAM, service accounts, encryption, Secret Manager, VPC Service Controls, and data access management.
- Familiarity with Git, CI/CD pipelines, Terraform or other Infrastructure as Code, automated testing, logging, monitoring, and deployment practices.
- Preferred experience with Cloud Monitoring, Cloud Logging, or equivalent observability tools.
- Added advantage: exposure to Looker, Looker Studio, Power BI, or Tableau.
- Strong analytical, troubleshooting, problem-solving, communication, ownership, and cross-functional collaboration skills.
- Ability to translate business requirements into maintainable technical solutions and work effectively in distributed delivery teams.
Tech Stack
Categories
Data Engineering
About Accenture
Accenture is a global professional services firm providing management consulting, systems integration and technology, cybersecurity, and business process outsourcing for enterprises and governments. It operates a services-driven model delivering projects and managed services, often with major cloud and software partners, across industries worldwide. Headquartered in Dublin and publicly traded on the NYSE (ACN), it originated as Andersen Consulting and adopted the Accenture name in 2001.