
Staff Data Engineer (AWS, Snowflake, Big Data)
Commonwealth Bank of Australia3 days ago
Sydney, AustraliaStaff+
Responsibilities
- Provide Staff-level technical leadership across data engineering, architecture, and full-cycle delivery.
- Design, build, deploy, observe, and optimize large-scale enterprise data platforms and solutions on AWS.
- Build production-grade batch, streaming, and near-real-time data pipelines, services, and engineering solutions using Python and Spark.
- Support credit-risk model development, validation, implementation, monitoring, orchestration, and SageMaker-based ML workloads.
- Modernize legacy data ecosystems and migrate warehouse technologies toward Snowflake and AWS-native solutions.
- Embed data controls, lineage, auditability, quality, reconciliation, and reliability into model-driven systems.
- Develop distributed data-processing and storage solutions using Hadoop, Spark, MapReduce, Kafka, and ETL frameworks.
- Improve engineering productivity through AI-powered tooling, ETL automation, and AI-assisted development.
Requirements
- Deep hands-on experience building large-scale enterprise data platforms on AWS.
- Strong proficiency in Python, PySpark or Spark, and SQL for production data pipelines and complex data processing.
- Experience supporting end-to-end model development lifecycles and building batch, streaming, and near-real-time data solutions.
- Experience with enterprise data architecture, scalable solution design, systems integration, data mesh architectures, data products, and data modeling.
- Experience modernizing data ecosystems, including migration from Teradata or legacy warehouses to Snowflake and AWS-native solutions.
- Experience with Hadoop-based distributed processing, including Spark, MapReduce, Sqoop, Kafka, and ETL frameworks.
- Strong Unix/Linux shell scripting and programming skills in Scala, Java, or Python.
- Experience with data warehousing technologies such as Snowflake, Teradata, Hadoop/Cloudera, Hive, and Oracle.
- Experience with DevOps and infrastructure automation tools including CI/CD, GitHub Actions, CloudFormation, and AWS CDK.
- Practical experience applying AI-assisted engineering, Generative AI concepts, and engineering productivity techniques.
- Cloudera CDP, Hadoop, Spark, Snowflake, or AWS Data Engineer certification is preferred.
Benefits
- Flexible work locations and arrangements, including changing start and finish times, part-time arrangements, and job sharing.
- Employees are expected to spend at least half of each month connecting in the office.
- Inclusive and respectful workplace supporting diverse perspectives, ideas, initiatives, and energy.
- Advertising end date: 03/12/2026.
Tech Stack
Amazon RedshiftApache FlinkApache HadoopApache HiveApache KafkaApache SparkAWSGitHub ActionsJavaLinuxMongoDBPostgreSQLPythonScalaSnowflakeSQL
Categories
Data Engineering
About Commonwealth Bank of Australia
Commonwealth Bank of Australia is an ASX-listed bank headquartered in Sydney that provides retail, business, and institutional banking, payments, and wealth services across Australia. Its offerings include transaction and savings accounts, home and business lending, cards, merchant services, digital banking, superannuation, and investment platforms such as CommSec. Founded in 1911, the bank serves millions of customers through branches, contact centres, and online channels.