DE - Cloud - Senior Data Platform Engineer - GDSN02
Ernst and Young4 days ago
Chennai, IndiaMid Level / Senior
Responsibilities
- Architect and manage Databricks and Snowflake infrastructure on Azure and AWS.
- Establish data governance frameworks covering access controls, lineage, compliance, and multi-cloud environments.
- Design and implement automated DataOps and MLOps pipelines for ingestion, transformation, model training, deployment, and monitoring.
- Automate infrastructure provisioning and configuration for workspaces, clusters, and supporting resources.
- Integrate data and ML lifecycles with CI/CD and source-control platforms.
- Implement monitoring, logging, alerting, security controls, and compliance checks for cloud data workloads.
- Optimize Databricks performance and cost, including autoscaling, spot instances, and job scheduling.
- Lead incident response, root cause analysis, troubleshooting, and reliability improvements.
- Collaborate with data engineering, analytics, ML, and client teams; document architectures and best practices.
- Mentor teams, create Golden Paths for developer workflows, and drive DevSecOps and platform adoption.
- Evaluate vendor packages and proposals, including business and technology fit, pricing, and support.
- Design AI-infused DevOps and platform engineering frameworks and participate in people initiatives.
Requirements
- BE, B.Tech, or MCA with 4–8 years of industry experience.
- Cloud Computing experience in multi-cloud Azure and AWS environments.
- Hands-on Databricks administration on Azure and/or AWS, including workspace, cluster, and job management.
- Experience establishing CI/CD pipelines with Azure DevOps, GitHub Actions, or GitLab CI.
- Strong understanding of DataOps and MLOps practices, including orchestration, monitoring, and governance.
- Hands-on Docker and Kubernetes experience, including AKS, EKS, or GKE.
- Scripting experience with PowerShell, Python, Groovy, and shell scripting, plus cloud-native CLIs.
- Experience with infrastructure as code using Terraform, ARM Templates, or Bicep.
- Experience with configuration management tools such as Ansible, Chef, or Puppet.
- Expertise creating Golden Paths for end-to-end developer workflows and automating applications and infrastructure.
- Strong knowledge of cloud networking, IAM, architecture strategy, cost management, security, compliance, monitoring, and troubleshooting.
- Preferred certifications include Microsoft Azure, AWS DevOps or Solutions Architecture, HashiCorp Terraform, and Certified Kubernetes Administrator certifications.
- Preferred familiarity with Terraform Cloud, Ansible Tower, GitHub Actions, GitHub Copilot, regulatory compliance, and recently developed technologies.
- Strong communication, stakeholder management, analytical, problem-solving, mentoring, research, presentation, and leadership skills; ability to travel as required.
Benefits
- Continuous learning, coaching, leadership development, and tools and flexibility to make a meaningful impact.
- Inclusive and diverse culture with collaboration across EY teams, service lines, geographies, and sectors.
- Global Delivery Services environment spanning Argentina, China, India, the Philippines, Poland, and the UK.
- Travel is required in accordance with client and job requirements.
Tech Stack
Categories
Data EngineeringDevOps
About Ernst and Young
Ernst & Young (EY) provides audit/assurance, tax, consulting, strategy and transactions services to enterprises, financial institutions, and public‑sector clients. Structured as a global network of partner‑owned member firms, it sells professional services on a fee basis, including a dedicated Financial Services Organization for banking, insurance, and capital markets. Headquartered in London, EY was formed in 1989 from the merger of Ernst & Whinney and Arthur Young, and operates in 150+ countries.