
Databricks Platform Engineer, AI and Data, Technology Consulting
Ernst and Young23 hours ago
Canberra, AustraliaSenior
Responsibilities
- Design, build, optimize, and maintain scalable Databricks-based cloud data platforms across Azure and AWS.
- Configure and administer Databricks workspaces, clusters, jobs, workflows, permissions, and platform services.
- Implement lakehouse architectures using Databricks, Apache Spark, Delta Lake, Unity Catalog, and cloud-native services.
- Develop infrastructure-as-code, CI/CD pipelines, and automation for repeatable platform delivery.
- Implement security, access controls, governance, monitoring, observability, and operational standards across cloud and Databricks environments.
- Collaborate with data engineers, AI engineers, architects, and stakeholders on ingestion, transformation, orchestration, storage, and consumption platform patterns.
- Troubleshoot Databricks, Spark, and cloud-platform issues, resolve incidents, and improve system stability and performance.
- Document platform designs, configurations, operating procedures, and technical standards.
- Support and guide team members and contribute to delivery planning and technical direction.
Requirements
- Strong experience designing and implementing cloud-based data platforms on Azure and/or AWS.
- Hands-on experience with the Databricks Data Intelligence Platform, including workspace configuration, compute management, jobs, workflows, and administration.
- Proficiency with Databricks, Apache Spark, Delta Lake, Unity Catalog, and related data engineering technologies.
- Experience supporting secure and governed data platforms, including identity and access management, permissions, secrets, networking, and data governance controls.
- Knowledge of infrastructure-as-code and DevOps practices, including Terraform, Git-based workflows, CI/CD pipelines, and automated deployment processes.
- Experience implementing platform security, governance, monitoring, and optimization frameworks.
- Strong problem-solving and analytical skills and the ability to resolve technical issues pragmatically.
- Excellent communication skills with technical and non-technical stakeholders.
- Experience contributing to or leading delivery in teams involving engineers, architects, and stakeholders.
- Experience with AI/ML platforms, model serving, MLOps, observability, monitoring, or optimization tools is advantageous.
- Experience with Power BI or Tableau is advantageous.
- Databricks certifications and enterprise-scale lakehouse implementation experience are highly regarded.
Benefits
- Hybrid position based in Canberra – Ngambri.
- Potential flexible work arrangements with reduced hours.
- Flexible work policies supporting work-life balance and autonomy.
- Yearly wellness incentive.
- Up to 8 additional weeks of flex leave per year.
- Family-friendly policies, including up to 26 weeks of gender-neutral paid parental leave.
- Career development with future-focused skills and world-class experiences.
- Inclusive and equitable recruitment and workplace environment.
About Ernst and Young
Ernst & Young (EY) provides audit/assurance, tax, consulting, strategy and transactions services to enterprises, financial institutions, and public‑sector clients. Structured as a global network of partner‑owned member firms, it sells professional services on a fee basis, including a dedicated Financial Services Organization for banking, insurance, and capital markets. Headquartered in London, EY was formed in 1989 from the merger of Ernst & Whinney and Arthur Young, and operates in 150+ countries.