Principal Data Engineer – Enterprise Data Products
Willis Towers Watson23 hours ago
Ipswich, United KingdomStaff+
Responsibilities
- Design and build scalable, resilient, and maintainable enterprise data solutions using Databricks, Apache Spark/PySpark, Python, SQL, and lakehouse technologies.
- Develop reusable patterns for data ingestion, transformation, integration, delivery, incremental processing, schema evolution, reprocessing, performance, and cost optimization.
- Contribute hands-on expertise to solution architecture, technical design, prototyping, implementation, and technology transformation.
- Apply automated testing, source control, CI/CD, observability, data quality, lineage, and appropriate engineering controls.
- Use AI-native engineering practices across design, coding, testing, troubleshooting, and operational automation.
- Work across data, application, API, authentication, platform, and cloud integration technologies to deliver end-to-end solutions.
- Investigate complex data, integration, and platform problems and identify pragmatic solutions, risks, dependencies, and trade-offs.
- Support engineers through pairing, code and design reviews, coaching, working examples, and knowledge sharing without direct line-management responsibility.
- Influence engineering standards, reusable frameworks, technical decisions, estimates, and incremental delivery approaches.
- Explore and prototype technologies and approaches that improve engineering effectiveness and Enterprise Data Products.
Requirements
- Proven and extensive experience in data engineering and significant experience delivering complex enterprise data engineering solutions.
- Deep hands-on experience with Databricks across implementation, development, production support, and long-term maintenance.
- Strong hands-on experience developing production-quality Databricks notebooks using Python and SQL.
- Demonstrable experience using AI as an integral part of software or data engineering, including design, coding, testing, and troubleshooting, with knowledge of safe and effective controls.
- Strong knowledge of modern data engineering, distributed processing, Lakehouse concepts, data modeling, ingestion, transformation, and integration patterns.
- Experience with Apache Spark or comparable large-scale data technologies and cloud-based data platforms and products.
- Experience with automated testing, source control, CI/CD, troubleshooting, technology modernization, migration, or architectural transformation.
- Ability to translate high-level or evolving requirements into pragmatic solutions and initial estimates.
- Experience influencing and supporting engineers without direct line-management responsibility.
- Hands-on experience with other lakehouse platforms such as Microsoft Fabric is desirable.
- Experience with Azure data platforms and cloud services, established enterprise data estates, data governance, metadata, lineage, observability, data quality, reusable engineering frameworks, modern APIs, integrations, and web application technologies is desirable.
Benefits
- 25 days of annual leave plus an additional WTW day.
- Private healthcare, life insurance, group income protection, and regular health assessments.
- Defined contribution pension scheme with company-matched contributions up to 10%.
- Hybrid working options, an employee assistance programme, and a fully paid volunteer day.
- Optional electric vehicle car scheme, share scheme, cycle-to-work programme, dental and optical cover, critical illness protection, and other perks.
Tech Stack
Categories
Data Engineering
About Willis Towers Watson
WTW is a global advisory, broking, and software firm serving employers, insurers, and institutional clients across people, risk, and capital. It provides insurance brokerage and risk advisory, employee benefits and pensions consulting, and HR/benefits administration software and analytics. Headquartered in London and listed on NASDAQ (WTW), the company was formed in 2016 via the Willis Group–Towers Watson merger and operates in roughly 140 markets worldwide.