15 days ago
Bengaluru, IndiaStaff+
Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines using Cloud Dataflow, Cloud Composer, Dataform/dbt, and Cloud Functions.
- Build real-time streaming pipelines using Cloud Pub/Sub, Kafka, and Dataflow.
- Implement data quality checks, workflow monitoring, alerting, error handling, and pipeline troubleshooting.
- Architect and implement data lake and data warehouse solutions using Dataproc, BigQuery, Cloud Storage, and Cloud SQL.
- Design data models, partitioning and clustering strategies, dimensional models, fact tables, data marts, and aggregation layers.
- Manage data lifecycle, archival, retention, security, encryption, access control, governance, lineage, and impact analysis.
- Integrate data from databases, APIs, SaaS applications, file systems, third-party providers, and external feeds, including CDC solutions.
- Support self-service analytics through data cataloging, documentation, and optimized BigQuery datasets.
Requirements
- Bachelor’s degree in computer science, engineering, or a related field.
- At least 6 years of experience in data engineering or related roles.
- At least 4 years of hands-on experience with Google Cloud Platform and services including BigQuery, Cloud Dataflow, Cloud Composer, Cloud Storage, Cloud Pub/Sub, Dataform/dbt, and Cloud Functions.
- Proficiency in Python or Java and SQL, with experience developing using Spark or Apache Beam.
- Strong understanding of Apache Beam, Apache Airflow, and distributed computing concepts.
- Experience with Cloud SQL, PostgreSQL, MySQL, Bigtable, and Firestore.
- Experience with Terraform, Cloud Deployment Manager, or similar infrastructure-as-code tools.
- Experience with Git and CI/CD pipelines, data modeling, and dimensional modeling.
- Preferred: Google Cloud Professional Data Engineer certification.
- Preferred: Docker, Kubernetes/GKE, data governance and compliance frameworks, Looker, Tableau, Power BI, Kafka, and Spark Streaming experience.
Tech Stack
Apache AirflowApache BeamApache KafkaApache SparkdbtDockerGitGoogle BigQueryGoogle Cloud PlatformJavaKubernetesMySQLPostgreSQLPythonSQLTerraform
Categories
Data Engineering
About Equinix
Equinix is a public digital infrastructure company that operates a global platform of colocation and interconnection data centers for enterprises, cloud and network providers. It sells space, power, and connectivity, plus services such as Equinix Fabric, Network Edge, and Equinix Metal for hybrid multicloud and edge deployments. Founded in 1998 and headquartered in Redwood City, California, it is listed on NASDAQ (EQIX) and runs hundreds of IBX facilities across major metros worldwide.