Commonwealth Bank of Australia

Principal Data Engineer (Snowflake, AWS, Python)

Commonwealth Bank of Australia
Apply
3 days ago
Sydney, Australia or Melbourne, AustraliaStaff+

Responsibilities

  • Architect and build scalable enterprise data platforms and production-grade data pipelines across AWS, Snowflake, Python, Spark, Kafka, and real-time streaming systems.
  • Design batch, streaming, and near-real-time data solutions using event-driven architectures and distributed processing.
  • Own solutions end to end from problem definition and architecture through engineering, deployment, observability, and optimization.
  • Modernize legacy data ecosystems and support warehouse migrations, including Teradata to Snowflake.
  • Set engineering standards, make pragmatic architectural decisions, mentor engineers, and raise technical capability across teams.
  • Apply AI-assisted engineering and explore AI-ready data capabilities to improve engineering productivity and customer experiences.

Requirements

  • Principal-level experience providing technical leadership across data engineering, architecture, and full-cycle delivery.
  • Deep hands-on experience building large-scale enterprise data platforms on AWS.
  • High proficiency in Python and experience building production-grade data pipelines, services, and engineering solutions.
  • Strong experience with Kafka, real-time messaging, streaming, and event-driven architectures.
  • Experience architecting batch, streaming, and near-real-time solutions across complex environments.
  • Experience modernizing data ecosystems, including migrations from legacy warehouse technologies to Snowflake.
  • Experience with enterprise data architecture, scalable solution design, systems integration, data modeling, data quality, and complex data pipelines.
  • Experience with several of the listed cloud, data platform, distributed processing, streaming, DevOps, AI-assisted engineering, and Microsoft ecosystem technologies.

Tech Stack

Amazon DynamoDBAmazon RedshiftApache FlinkApache HadoopApache HiveApache KafkaApache SparkAWSDatabricksGitHub ActionsPythonScalaSnowflakeSQL

Categories

Data Engineering
Commonwealth Bank of Australia

About Commonwealth Bank of Australia

10,000+ employees

Commonwealth Bank of Australia is an ASX-listed bank headquartered in Sydney that provides retail, business, and institutional banking, payments, and wealth services across Australia. Its offerings include transaction and savings accounts, home and business lending, cards, merchant services, digital banking, superannuation, and investment platforms such as CommSec. Founded in 1911, the bank serves millions of customers through branches, contact centres, and online channels.

Contact me