1 day ago
Pune, India or Bengaluru, IndiaSenior
Responsibilities
- Design and implement scalable batch and streaming data pipelines using Python and GCP services.
- Build and maintain Dataform SQLX transformation workflows integrated with BigQuery.
- Develop event-driven and real-time processing solutions using Pub/Sub, Cloud Functions, and Cloud Run.
- Manage structured and unstructured data in GCS, BigQuery, and Cloud Spanner.
- Develop reusable Python modules, utilities, and SDKs for internal data platform tooling.
- Provision and manage GCP infrastructure with Terraform.
- Maintain GitLab pipelines for automated testing, linting, and deployment of data workloads.
- Manage Cloud Run containerized workloads and related Dockerfiles and container registries.
- Apply unit testing, monitoring, alerting, observability, and code review practices.
Requirements
- At least three years of experience in Google Cloud Platform architecture.
- Five or more years of data engineering experience, including at least two years working with GCP.
- Strong Python programming skills, including object-oriented programming, packaging, and performance optimization.
- Hands-on BigQuery experience covering query optimization, partitioning, clustering, and data modeling.
- Experience with Pub/Sub, Cloud Run and/or Cloud Functions, Terraform, GitLab CI/CD, pytest, and GCS data lake patterns.
- Experience with file formats including Parquet, Avro, and JSON.
- Minimum 15 years of full-time education.
- Preferred qualifications include Dataform, Cloud Spanner, data mesh or data lakehouse architecture, Apache Beam, Dataflow, GCP Professional Data Engineer or Cloud Architect certification, Apache Airflow, and Cloud Composer experience.
Benefits
- The position is based at the employer's Bengaluru office.
Tech Stack
Apache AirflowApache BeamDockerGoogle BigQueryGoogle CloudGoogle Cloud PlatformpytestPythonTerraform
Categories
Data Engineering
About Accenture
Accenture is a global professional services firm providing management consulting, systems integration and technology, cybersecurity, and business process outsourcing for enterprises and governments. It operates a services-driven model delivering projects and managed services, often with major cloud and software partners, across industries worldwide. Headquartered in Dublin and publicly traded on the NYSE (ACN), it originated as Andersen Consulting and adopted the Accenture name in 2001.