Databricks Platform Engineer, AI and Data, Technology Consulting
Ernst and Young11 days ago
Melbourne, AustraliaSenior
Responsibilities
- Design, build, optimize, and maintain scalable Databricks-based data platforms across Azure and AWS.
- Configure and administer Databricks workspaces, clusters, jobs, workflows, permissions, and platform services.
- Implement Lakehouse architectures using Databricks, Apache Spark, Delta Lake, Unity Catalog, and cloud-native services.
- Develop infrastructure-as-code, CI/CD pipelines, and automation for repeatable platform delivery.
- Implement security, access controls, governance, monitoring, observability, and operational standards.
- Collaborate with data engineers, AI engineers, architects, and stakeholders on platform patterns for ingestion, transformation, orchestration, storage, and consumption.
- Troubleshoot Databricks, Spark, and cloud platform issues, resolve incidents, and improve stability and performance.
- Document platform designs, configuration patterns, operational processes, and technical standards.
- Support team members and contribute to delivery planning and technical direction.
Requirements
- Strong experience designing and implementing cloud-based data platforms on Azure and/or AWS.
- Hands-on experience with the Databricks Data Intelligence Platform, including workspace configuration, compute management, jobs, workflows, and administration.
- Proficiency with Databricks, Apache Spark, Delta Lake, Unity Catalog, and related data engineering technologies.
- Experience supporting secure and governed data platforms, including identity and access management, permissions, secrets, networking, and data governance controls.
- Knowledge of infrastructure-as-code and DevOps practices, including Terraform, Git-based workflows, CI/CD pipelines, and automated deployments.
- Experience implementing platform security, governance, monitoring, and optimization frameworks.
- Strong problem-solving, analytical, communication, and stakeholder-engagement skills.
- Experience contributing to or leading delivery in teams involving engineers, architects, and business stakeholders.
- Experience with AI/ML platforms, model serving, and MLOps in Databricks is preferred.
- Experience with platform observability, monitoring, optimization, Power BI or Tableau is preferred.
- Databricks certifications and enterprise-scale Lakehouse implementation experience are highly regarded.
Benefits
- Hybrid work arrangement based in Melbourne, Brisbane, or Sydney, with potential flexibility for reduced hours.
- Flexible work policies supporting work-life balance and autonomy.
- Yearly wellness incentive and access to up to 8 additional weeks of flex leave per year.
- Up to 26 weeks of gender-neutral paid parental leave and family-friendly policies.
- Career development with future-focused skills and globally connected experiences.
- Inclusive and equitable recruitment and workplace environment.