over 1 year ago
Bengaluru, IndiaStaff+
Responsibilities
- Lead migration of large-scale datasets and legacy on-premises or hybrid data warehouse environments to Google BigQuery.
- Design data migration strategies focused on data quality, integrity, and performance.
- Integrate structured and unstructured data from APIs, relational databases, IoT devices, and telemetry sources.
- Build real-time streaming pipelines for large-scale data ingestion and processing.
- Modernize SSIS packages into cloud-native ETL pipelines using Apache Airflow, Python, Spark, and GCP-native tools.
- Develop reusable data transformations using Python, Java, Spark, and SQL.
- Write and optimize BigQuery SQL queries, stored procedures, and scheduled jobs.
- Monitor and optimize query performance and cloud data processing costs.
Requirements
- 5+ years of data engineering experience focused on cloud and big data technologies.
- At least 2 years of hands-on experience with GCP and BigQuery.
- Proven experience migrating on-premises data systems to the cloud.
- Strong development experience with Apache Airflow, Python, and Apache Spark.
- Experience with streaming data ingestion, particularly IoT or sensor data.
- Strong SQL development and BigQuery performance-tuning skills.
- Understanding of cloud architecture, data modeling, and data warehouse design.
- Familiarity with Git and CI/CD practices for managing data pipelines.
- Preferred: GCP Professional Data Engineer certification.
- Preferred: experience with dbt, Kafka, Terraform, ML pipelines, analytics engineering, or DataOps/DevOps methodologies.
Benefits
- Remote flexibility.
- Competitive compensation and benefits.
- Growth opportunities.
- Opportunity to lead large-scale cloud transformation initiatives.
Tech Stack
Apache AirflowApache KafkaApache SparkdbtGitGoogle BigQueryGoogle Cloud PlatformJavaMicrosoft SQL ServerPythonSQLTerraform
Categories
Data Engineering
About Brillio
Brillio is a privately held digital engineering and enterprise AI services firm that helps Fortune 1000 companies modernize products, data, and infrastructure. It combines consulting and managed services with an accelerator platform (ADAM) across cloud, AI, data engineering, customer experience, and infrastructure workstreams. Founded in 2014 and headquartered in Dallas, it operates delivery centers across North America, Europe, and Asia, with focus areas that include healthcare and life sciences.