Senior Data Engineer
Hack The Box6 months ago
Remote, EMEA +4 moreSenior
Responsibilities
- Design and build batch and streaming pipelines using Dataflow, Pub/Sub, Kafka, BigQuery, Bigtable, and ClickHouse.
- Drive migration from Snowflake to the GCP-native data platform and retire legacy or shadow pipelines.
- Own Airflow orchestration, including SLAs, retries, and data quality gates.
- Model and serve data for analytics, ML training, feature stores, and low-latency online inference.
- Partner with ML engineers on feature pipelines, data drift monitoring, and retraining workflows.
- Capture stakeholder requirements and translate them into pragmatic data products.
- Improve data quality, reliability, observability, and cost efficiency.
- Identify and integrate valuable third-party and other data sources.
- Build and consume APIs and treat infrastructure as code.
Requirements
- Strong data modeling and warehouse architecture skills, including dimensional modeling, event-driven patterns, and lakehouse patterns.
- Hands-on experience with GCP data services; BigQuery is required, while Pub/Sub, Dataflow, Bigtable, and Cloud Composer are strong pluses.
- Production experience building streaming pipelines with Dataflow/Beam, Flink, or Spark Structured Streaming using Kafka and/or Pub/Sub.
- Strong Python and SQL skills with experience writing production-quality code.
- Production experience with ClickHouse or another columnar OLAP engine.
- Experience with Airflow or an alternative such as Prefect or Dagster.
- Experience with dbt or an equivalent transformation framework.
- Experience migrating legacy warehouses such as Snowflake, Redshift, or Synapse to cloud-native platforms is a plus.
- Working knowledge of production ML, including feature engineering, feature stores, model deployment, drift monitoring, and retraining.
- Experience with Docker and Kubernetes, plus CI/CD and infrastructure-as-code practices.
- Experience with CDC tooling such as Datastream or Debezium is a bonus.
- Experience with Vertex AI or Feature Store is a bonus.
Benefits
- Private healthcare.
- Paid paternity leave.
- 25 annual leave days.
- Free lunch and snacks at the office.
- €120 Ticket Restaurant benefit through Edenred.
- Dedicated training and professional development budget, including conference participation.
- Full access to Hack The Box lab offerings.
- State-of-the-art equipment, including a Mac, iPhone, and mobile plan.
- Fully remote work across Europe, or hybrid work in Athens with two office days and three remote days plus one month of work from anywhere; Greece-based employees beyond 55 km from the Athens office may work fully remotely.
- Relocation support includes a 10% relocation bonus, a 50% income tax reduction, and accommodation during the first few weeks.
- Benefits may vary by location and employment arrangement, including UK, Greece, or Employer of Record engagement.
Tech Stack
Amazon RedshiftApache AirflowApache BeamApache FlinkApache KafkaApache SparkClickHousedbtDockerGitHub ActionsGoogle BigQueryGoogle Cloud PlatformKubernetesPythonSnowflakeSQL
Categories
Data Engineering
About Hack The Box
Hack The Box builds an online cybersecurity training and readiness platform with hands-on labs, cyber ranges, and live-fire simulations for individuals and enterprise security teams. Organizations use it for upskilling, skills assessment, and certifications across offensive and defensive disciplines. Founded in 2017 and headquartered in Folkestone, UK, the privately held company serves enterprises, government agencies, and MSSPs, with additional offices in the US, Australia, and Greece.