Principal Data Engineer
Simple Machines6 months ago
London, United KingdomStaff+
Responsibilities
- Own the architecture of modern, cloud-native data platforms and make decisions across ingestion, storage, processing, and access layers.
- Design and deliver greenfield data platforms using Databricks, Snowflake, AWS, and GCP.
- Build and optimize batch, real-time, streaming, and event-driven data pipelines.
- Orchestrate data workflows and process large datasets for production-scale performance.
- Work across relational, NoSQL, and analytical storage technologies and optimize storage formats and access patterns.
- Implement secure, compliant, governed data solutions with security by design.
- Consult directly with clients and translate business needs into pragmatic engineering solutions.
- Set engineering standards, review designs and code, mentor engineers, and provide technical direction.
- Improve data quality, testing, observability, and operational excellence across teams.
Requirements
- Strong Python and SQL skills.
- Deep experience with Spark and modern data platforms such as Databricks and Snowflake.
- Solid experience with cloud data services on AWS or GCP.
- Demonstrated ownership of large-scale data platform architectures and strong data modeling skills.
- Experience building and operating large-scale production data pipelines.
- Experience with multiple storage technologies and data formats.
- Infrastructure-as-code experience with Terraform or Pulumi.
- Experience with CI/CD tools such as GitHub Actions or ArgoCD.
- Experience with data testing and quality frameworks such as dbt, Great Expectations, or Soda.
- Experience in consulting or professional services environments.
- Ability to advise clients, challenge assumptions, mentor senior engineers, and influence technical culture.
Benefits
- Hybrid work arrangement in London, United Kingdom.
- Full-time employment.
- Opportunity to work on high-impact problems and build modern platforms rather than maintain legacy systems.
- Autonomy, influence, and room for professional growth.
- Work alongside experienced senior engineers.
Tech Stack
Apache AirflowApache CassandraApache FlinkApache KafkaApache SparkAWSDatabricksdbtGitHub ActionsGoogle BigQueryGoogle Cloud PlatformMongoDBPostgreSQLPythonSnowflakeSQLTerraform
Categories
Data Engineering
About Simple Machines
Simple Machines is a privately held consulting and engineering firm that designs and builds data platforms, machine-learning/AI systems, and bespoke software for enterprises and public-sector clients. The company provides architecture, data engineering, and scalable distributed systems expertise across AWS, GCP, Azure, Databricks, Snowflake, and Confluent. Founded in 2008 and headquartered in Sydney, it operates globally with teams in Australia, the UK, Europe, and New Zealand.