Experian

Lead Data Platform Engineer - AWS, Spark

Experian
Apply
2 days ago
Nottingham, United KingdomStaff+

Responsibilities

  • Lead and support an agile technical team delivering fraud-related data platform solutions.
  • Guide the delivery of AWS-based data pipelines, data lake components, and producer-side data platform capabilities.
  • Build, test, maintain, and enhance data pipelines from Snowflake and internal product and data platforms.
  • Support ownership and handover of pipelines and cloud components developed by the central Data Analytics Platform team.
  • Prepare and publish metadata so data assets can be surfaced through the data catalogue.
  • Champion DevSecOps, CI/CD, Infrastructure as Code, automation, and cloud-native tooling.
  • Mentor and coach senior and junior engineers while promoting innovation, learning, collaboration, and modern engineering practices.
  • Contribute to technical roadmap discussions and collaborate with central platform teams on shared requirements.

Requirements

  • Strong experience building data pipelines, data lake solutions, or data platform capabilities.
  • Strong experience with AWS cloud services relevant to data engineering and platform delivery.
  • Strong experience with Terraform, CI/CD pipelines, Git, and Jenkins.
  • Strong experience developing with Apache Spark or Hadoop.
  • Significant programming experience using Scala or Python.
  • Experience with Configuration as Code principles and API integration.
  • Strong understanding of cloud security, data privacy, access control, and governance in data platform environments.
  • AWS Developer Certification or another relevant AWS certification is desirable.
  • Strong Terraform or CloudFormation expertise is desirable.
  • Experience with OpenShift or Kubernetes is desirable.
  • Experience with automated testing tools such as pytest or Scalatest is desirable.
  • Familiarity with SonarQube, Veracode, governed or regulated data environments, data catalogues, metadata management, data marketplaces, or federated data mesh concepts is desirable.

Benefits

  • Great compensation package and discretionary bonus plan.
  • 25 days of annual leave, 8 bank holidays, and 3 volunteering days.
  • Option to purchase additional annual leave.
  • Pension, Bupa healthcare, and Sharesave scheme.
  • Hybrid Nottingham-based work arrangement with 40% office attendance.
  • Inclusive, purpose-driven culture and workplace accommodations for disabilities or special needs.

Tech Stack

Apache HadoopApache SparkAWSGitJenkinsKubernetesOpenShiftpytestPythonScalaSnowflakeSonarQubeTerraform

Categories

Data Engineering
Experian

About Experian

10,000+ employees

Experian builds credit reporting, identity and fraud prevention, and data analytics software that help lenders, insurers, healthcare providers, and marketers make decisions; it also offers consumer credit monitoring and scores. The company earns via data licensing, decisioning software, and subscription services, operates in 30+ countries, is headquartered in Costa Mesa, California, and is a FTSE 100 public company listed on the London Stock Exchange (EXPN); it is one of the three major U.S. credit bureaus.

Contact me