Senior Data Platform Engineer
Bits In Glass1 month ago
Remote, United Kingdom or London, United KingdomSenior
Responsibilities
- Design, build, and maintain scalable data platforms on Google Cloud Platform for financial-sector clients.
- Develop and optimize batch and streaming data pipelines using Dataflow, Dataproc, Pub/Sub, and Cloud Composer.
- Architect data lake, data warehouse, and lakehouse solutions, including data modeling and performance tuning in BigQuery.
- Provision and manage data infrastructure using Terraform and CI/CD pipelines.
- Implement data governance, lineage, quality, and cataloging frameworks for regulatory and client requirements.
- Apply security practices including IAM, encryption, KMS, data masking, and access controls across data environments.
- Monitor, troubleshoot, and resolve platform performance, reliability, and data quality issues.
- Collaborate with client teams, architects, and internal stakeholders to translate business needs into technical solutions.
- Mentor engineers and promote DataOps and engineering best practices.
- Communicate architecture decisions, progress, and analysis to technical and non-technical audiences.
Requirements
- 7+ years of experience in data engineering or data platform engineering, including at least 3 years in a senior or lead capacity.
- Strong hands-on experience building and operating data platforms on Google Cloud Platform.
- Experience with BigQuery, Dataflow, Dataproc, Cloud Composer or Airflow, Pub/Sub, Cloud Storage, Bigtable, and Cloud SQL.
- Proficiency in Python and SQL, with experience in Spark or Apache Beam.
- Experience designing data warehouse and data lake architectures, including data modeling.
- Experience with Terraform and CI/CD tooling such as Jenkins, GitHub, or Cloud Build.
- Strong understanding of data security in regulated environments.
- Knowledge of DevOps, DataOps, and Agile delivery practices.
- Strong written and verbal communication, analytical, debugging, and problem-solving skills.
- Preferred qualifications include GCP Professional Data Engineer or Professional Cloud Architect certification and experience in banking, insurance, or other regulated financial services.
- Preferred experience includes Dataplex, Data Catalog, dbt, Great Expectations, Docker, Kubernetes or GKE, Snowflake, Databricks, Kafka, and Vertex AI pipelines.
Tech Stack
Apache AirflowApache BeamApache KafkaApache SparkDatabricksdbtDockerGoogle BigQueryGoogle Cloud PlatformJenkinsKubernetesPythonSnowflakeSQLTerraform
Categories
Data Engineering
About Bits In Glass
Bits In Glass is an IT consulting firm that designs and implements enterprise automation, integration, and CRM solutions using platforms like Appian, MuleSoft, Salesforce, Pega, and Blue Prism. It delivers project-based and managed services to large organizations across North America, Europe, and India, covering AI, cloud, data, and business applications. Founded in 2002 and headquartered in Edmonton, it is privately held and received private‑equity investment in 2024.