16 hours ago
Bengaluru, IndiaMid Level / Senior
Responsibilities
- Design and implement scalable batch and streaming data pipelines using Python and Google Cloud services.
- Build and maintain Dataform SQLX transformation workflows integrated with BigQuery.
- Develop event-driven and real-time processing solutions using Pub/Sub, Cloud Functions, and Cloud Run.
- Manage structured and unstructured data in GCS, BigQuery, and Cloud Spanner.
- Develop reusable Python modules, utilities, and SDKs for internal data platform tooling.
- Write unit tests with pytest and participate in code reviews.
- Provision and manage GCP infrastructure using Terraform.
- Maintain GitLab CI/CD pipelines for testing, linting, and deployment of data workloads.
- Manage containerized Cloud Run workloads, including Dockerfiles and container registries.
- Support monitoring, alerting, and observability for data pipelines and GCP resources.
Requirements
- At least 3 years of experience in Google Cloud Platform Architecture.
- 5+ years of data engineering experience, including at least 2 years on GCP.
- Strong Python programming skills, including object-oriented programming, packaging, and performance optimization.
- Hands-on BigQuery experience including query optimization, partitioning, clustering, and data modeling.
- Experience with Pub/Sub, Cloud Run and/or Cloud Functions, pytest, Terraform, GitLab CI/CD, and GCS data lake patterns.
- Experience working with file formats such as Parquet, Avro, and JSON.
- Knowledge of data mesh or data lakehouse architecture patterns.
- Familiarity with Apache Beam, Dataflow, Apache Airflow, Cloud Composer, or similar orchestration tools.
- GCP Professional Data Engineer or Cloud Architect certification is preferred.
- Completion of 15 years of full-time education is required.
Benefits
- Position is based at the Accenture Bengaluru office.
Tech Stack
Apache AirflowApache BeamDockerGoogle BigQueryGoogle CloudGoogle Cloud PlatformpytestPythonTerraform
Categories
BackendData Engineering
About Accenture
Accenture is a global professional services firm providing management consulting, systems integration and technology, cybersecurity, and business process outsourcing for enterprises and governments. It operates a services-driven model delivering projects and managed services, often with major cloud and software partners, across industries worldwide. Headquartered in Dublin and publicly traded on the NYSE (ACN), it originated as Andersen Consulting and adopted the Accenture name in 2001.