1 month ago
Kochi, IndiaStaff+
Responsibilities
- Design, develop, and maintain scalable batch and real-time data pipelines using GCP services, BigQuery, Python, and SQL.
- Build ETL/ELT workflows for structured, semi-structured, and unstructured data from multiple source systems.
- Design and maintain BigQuery datasets, tables, views, materialized views, partitioning, clustering, and analytical data models.
- Optimize BigQuery queries, storage, processing workloads, pipeline architecture, performance, and cloud consumption costs.
- Develop curated data layers, reusable datasets, and data models for analytics, BI, machine learning, reporting, and downstream applications.
- Implement data validation, reconciliation, quality checks, error handling, lineage, monitoring, alerting, and operational controls.
- Orchestrate pipelines using Cloud Composer, Apache Airflow, Workflows, or equivalent technologies.
- Troubleshoot complex data and pipeline issues and support production stability and reliability.
- Collaborate with architects, data scientists, analysts, application teams, business stakeholders, and cross-functional teams.
- Provide technical guidance, conduct code reviews, establish development standards, and support junior engineers.
- Promote Git-based development, automated testing, CI/CD, Infrastructure as Code, documentation, monitoring, security, and production support practices.
Requirements
- 5–8 years of experience in data engineering, data warehousing, ETL/ELT development, SQL development, or large-scale data processing.
- Strong hands-on experience designing and implementing production data solutions on Google Cloud Platform.
- Strong production experience with BigQuery, including SQL development, partitioning, clustering, materialized views, query optimization, execution analysis, and cost management.
- Strong Python programming experience for data processing, pipeline development, automation, and integration.
- Advanced SQL skills, including joins, aggregations, CTEs, window functions, analytical queries, optimization, and troubleshooting.
- Experience designing and supporting production-grade batch and streaming data pipelines and scalable cloud data solutions.
- Hands-on experience with BigQuery, Google Cloud Storage, Dataflow, Pub/Sub, Dataproc, and Cloud Composer or Apache Airflow.
- Understanding of batch processing, streaming architecture, ETL/ELT patterns, data lakes, data warehouses, and cloud data platforms.
- Experience designing dimensional, analytical, and curated data models for reporting, BI, analytics, and machine learning use cases.
- Experience with Dataform, dbt, or similar SQL-based transformation frameworks is preferred.
- Experience with Cloud Functions, Cloud Run, Workflows, or other serverless GCP services is advantageous.
- Understanding of GCP security concepts including IAM, service accounts, encryption, Secret Manager, VPC Service Controls, and data access management.
- Familiarity with Git, CI/CD pipelines, Terraform, automated testing, logging, monitoring, and deployment practices.
- Experience with Cloud Monitoring, Cloud Logging, or equivalent observability tools is preferred.
- Exposure to Looker, Looker Studio, Power BI, or Tableau is advantageous.
- Strong analytical, troubleshooting, problem-solving, communication, ownership, and prioritization skills.
- Ability to translate business requirements into scalable technical solutions and work effectively with distributed delivery teams.
Tech Stack
Categories
Data Engineering
About Accenture
Accenture is a global professional services firm providing management consulting, systems integration and technology, cybersecurity, and business process outsourcing for enterprises and governments. It operates a services-driven model delivering projects and managed services, often with major cloud and software partners, across industries worldwide. Headquartered in Dublin and publicly traded on the NYSE (ACN), it originated as Andersen Consulting and adopted the Accenture name in 2001.