Staff Data Engineer
Careers at Eucalyptus2 days ago
London, United KingdomStaff+
Responsibilities
- Own complex, multi-sprint data platform initiatives including CDC connector buildouts, BigQuery warehouse workstreams, event-streaming infrastructure, lower environments, and shared standards.
- Design and maintain ingestion pipelines and platform infrastructure from source connectivity through raw and cleansed warehouse layers.
- Deploy and operate containerized services on Kubernetes (EKS) across AWS and GCP.
- Design and operate Kafka and Flink event-driven and streaming pipelines delivering CDC and user event tracking streams into BigQuery.
- Own CDC ingestion end to end, including Terraform provisioning, schema-change governance, production monitoring, and operational runbooks.
- Define and maintain data contracts, schema governance, service-level agreements, quality checks, anomaly detection, validation, and data-drift alerting.
- Establish KPIs and SLOs, build Datadog monitoring and alerting as code, participate in on-call rotations, and resolve Tier 1 tickets and root causes.
- Build platform capabilities that support Analytics Engineering, Data Science, and Machine Learning roadmaps while partnering on HIPAA/PHI, UK GDPR, cross-border access, and infrastructure hardening controls.
- Mentor Senior Data Engineers through design reviews, code reviews, pairing, and cross-squad standards adoption.
- Participate in architecture reviews and promote testing, CI/CD, observability as code, Airflow, dbt, and Schema Registry governance.
Requirements
- Typically 8+ years of experience designing, building, and operating production ingestion and streaming platforms at scale.
- Strong Python and SQL skills with practical experience using Kafka, Flink, dbt, Airflow, and BigQuery.
- Experience owning complex technical initiatives from scoping and design through production rollout across multiple teams.
- Experience setting SLOs, building monitoring and alerting as code, participating in on-call rotations, and writing operational runbooks.
- Experience with data contracts, schema governance, Schema Registry tooling, and pipeline data-quality checks.
- Comfort with Terraform and CI/CD patterns across AWS and GCP, including infrastructure hardening with DevOps and security teams.
- Experience mentoring engineers through design reviews, code reviews, pairing, and technical influence across squads.
- Ability to collaborate with Analytics Engineers, Data Scientists, Machine Learning Engineers, legal, and security stakeholders.
- Experience with Kotlin, Go, or similar for Kafka-ecosystem services is preferred.
- Experience in regulated sectors such as healthtech, fintech, or gaming is preferred.
- Familiarity with reverse-ETL or data activation tools such as Hightouch or Segment is preferred.
Benefits
- Employee options programme.
- Annual development budget plus three days of professional development leave.
- Hybrid work arrangement with three days per week in the Old Street office.
- Private health insurance through Vitality or a monthly wellness allowance.
- 25 days of holiday and enhanced parental leave.
- Option to work from anywhere for three weeks per year.
- Office socials, clubs, after-school sport, and seasonal events.
Tech Stack
Apache AirflowApache FlinkApache KafkaAWSDatadogdbtGoGoogle BigQueryGoogle Cloud PlatformKotlinKubernetesPythonSQLTerraform
Categories
Data Engineering