Data Engineer GCP (Google Cloud Plataform)
Act Digitalover 1 year ago
Porto, PortugalMid Level
Responsibilities
- Understand user and project requirements and communicate technical issues and solutions with architects, project managers, and peers.
- Write and update interface contracts and develop data pipelines based on defined architectures.
- Deploy requested infrastructure, particularly using Terraform.
- Conduct code reviews and coordinate peer reviews when merging code changes.
- Define and perform functional and technical pipeline tests, communicate results, and document testing outcomes.
- Present completed development during deployment reviews with the Data Architect and Lead DataOps.
- Monitor deployments, communicate errors, and apply deployment, logging, and monitoring strategies.
Requirements
- At least one year of experience with Google Cloud Platform and general knowledge of its services.
- At least two years of experience with Apache Airflow and knowledge of data orchestration.
- At least two years of extensive Google BigQuery experience, including table and query optimization and database architecture design.
- At least one year of experience with Terraform.
- General knowledge of Azure Cloud Platform is a plus.
- Apache Spark and PySpark experience is optional but highly valued.
- Knowledge of Pub/Sub, Kafka, Google Cloud Storage, and Dataflow or Apache Beam is a plus.
- Ability to understand business and technological challenges, communicate clearly, and apply technical and industry practices.
Benefits
- Remote position.
Tech Stack
Apache AirflowApache BeamApache KafkaApache SparkAzureGoogle BigQueryGoogle CloudGoogle Cloud PlatformTerraform
Categories
Data Engineering
About Act Digital
Act Digital is a global IT services and consulting group that designs and builds custom software, data, and cloud solutions for large enterprises. It provides end-to-end digital product services—strategy, UX, agile engineering, architecture, and data science—via project delivery and dedicated teams. The company is privately held, headquartered in Barueri, São Paulo, with offices across Brazil, North America, and Europe.