Staff Data Engineer
Commonwealth Bank of Australia23 hours ago
Sydney, AustraliaStaff+
Responsibilities
- Design and build scalable data platforms, pipelines, and data products using Python, PySpark, Scala, SQL, and distributed data processing.
- Integrate data from diverse sources into centralized data lakes and cloud platforms.
- Create technology blueprints and engineering roadmaps for long-term transformation.
- Define and evolve shared semantic models and metric definitions across products and domains.
- Drive adoption of data catalogues, glossaries, and stewardship practices for discoverability and self-service.
- Implement policy-as-code for data access, classification, and retention.
- Ensure data security, governance, compliance, and risk minimization.
- Implement automated testing, observability, and end-to-end lineage to improve data trust and incident resolution.
- Optimize ETL jobs, query performance, and cloud costs.
- Communicate with stakeholders and contribute to team-wide engineering practices.
Requirements
- AWS Certified Data Engineer Associate certification with demonstrable data engineering experience.
- Strong expertise in Python, PySpark, SQL, Spark, Scala, and distributed data processing.
- Experience with AWS Glue, Lambda, Redshift, Athena, Kinesis, and EMR.
- Experience with AWS DataZone, Lake Formation, IAM, and encryption.
- Experience using AI-powered coding assistants such as Cline, Roo, and Aider.
- Exposure to AWS Marketplace, API Gateway, and data monetization strategies.
- Experience with Terraform, CloudFormation, GitOps, and serverless frameworks.
- Experience with Snowflake or Apache Iceberg.
- Knowledge of data product thinking and Data Mesh principles.
- Ability to design secure, reliable, scalable, and business-focused data solutions.
Benefits
- Flexible working options, including changing start and finish times, part-time arrangements, and job sharing.
- At least half of each month connecting in the Sydney or Melbourne office.
- Accessibility support is available through HR Direct.
- Advertising end date: 22/11/2026.
Tech Stack
Categories
Data Engineering
About Commonwealth Bank of Australia
Commonwealth Bank of Australia is an ASX-listed bank headquartered in Sydney that provides retail, business, and institutional banking, payments, and wealth services across Australia. Its offerings include transaction and savings accounts, home and business lending, cards, merchant services, digital banking, superannuation, and investment platforms such as CommSec. Founded in 1911, the bank serves millions of customers through branches, contact centres, and online channels.