1 day ago
Bengaluru, IndiaStaff+
Responsibilities
- Design, develop, and maintain scalable ETL/ELT pipelines using Cloud Dataflow, Cloud Composer, Dataform/dbt, and Cloud Functions.
- Build real-time streaming pipelines using Cloud Pub/Sub, Kafka, and Dataflow.
- Implement data quality checks, workflow monitoring, alerting, error handling, and pipeline troubleshooting.
- Architect and implement data lake and data warehouse solutions using Dataproc, BigQuery, Cloud Storage, and Cloud SQL.
- Design data models, partitioning and clustering strategies, dimensional models, fact tables, data marts, and aggregation layers.
- Manage data lifecycle, archival, retention, security, encryption, and access control across storage layers.
- Support analytics and reporting through BigQuery datasets, data cataloging, documentation, lineage, and impact analysis.
- Integrate data from databases, SaaS applications, file systems, external feeds, and third-party data providers, including CDC solutions for real-time synchronization.
- Optimize pipeline performance and cost through resource allocation and scheduling.
- Develop infrastructure using Terraform, Cloud Deployment Manager, or similar tools.
Requirements
- Bachelor’s degree in computer science, engineering, or a related field.
- At least 4 years of experience in data engineering or related roles.
- At least 4 years of specific hands-on experience with Google Cloud Platform and services including BigQuery, Cloud Dataflow, Cloud Composer, Cloud Storage, Cloud Pub/Sub, Dataform/dbt, and Cloud Functions.
- Proficiency in Python or Java and SQL, with experience in Spark or Beam development.
- Strong understanding of Apache Beam, Apache Airflow, and distributed computing concepts.
- Experience with Cloud SQL, PostgreSQL, MySQL, Bigtable, and Firestore.
- Experience with Terraform, Cloud Deployment Manager, or similar infrastructure-as-code tools.
- Experience with Git and CI/CD pipelines, data modeling, and dimensional modeling.
- Preferred: Google Cloud Professional Data Engineer certification.
- Preferred: Docker and Kubernetes/GKE experience.
- Preferred: experience with data governance and compliance frameworks including GDPR, HIPAA, and SOX.
- Preferred: familiarity with Looker, Tableau, or Power BI.
- Preferred: experience with Kafka and Spark Streaming.
Benefits
- Equinix provides an inclusive workplace and reasonable accommodations for qualified individuals with disabilities.
- The role is a new position within the organization.
Tech Stack
Apache AirflowApache BeamApache KafkaApache SparkdbtDockerGitGoogle BigQueryGoogle Cloud PlatformJavaKubernetesMySQLPostgreSQLPythonSQLTerraform
Categories
Data Engineering
About Equinix
Equinix is a public digital infrastructure company that operates a global platform of colocation and interconnection data centers for enterprises, cloud and network providers. It sells space, power, and connectivity, plus services such as Equinix Fabric, Network Edge, and Equinix Metal for hybrid multicloud and edge deployments. Founded in 1998 and headquartered in Redwood City, California, it is listed on NASDAQ (EQIX) and runs hundreds of IBX facilities across major metros worldwide.