
Agile-DE - Cloud - Senior Data Platform Engineer - GDSN02
Ernst and Young26 days ago
Chennai, IndiaSenior
Responsibilities
- Architect and manage Databricks and Snowflake workspaces, clusters, jobs, and supporting infrastructure across Azure and AWS.
- Establish data governance frameworks covering access controls, lineage, compliance, and multi-cloud or on-premises environments.
- Design and implement DataOps and MLOps pipelines for ingestion, transformation, model training, deployment, monitoring, versioning, and orchestration.
- Automate infrastructure provisioning with Terraform, ARM Templates, or Bicep and integrate platforms with CI/CD pipelines.
- Implement monitoring, logging, alerting, incident response, root cause analysis, performance optimization, and cost optimization for Databricks workloads.
- Apply security, DevSecOps, secrets management, network isolation, encryption, vulnerability scanning, and compliance checks.
- Collaborate with data engineering, analytics, and machine learning teams; lead platform adoption and mentor teams on Databricks, DataOps, and MLOps practices.
- Document architectures and operating procedures, evaluate vendor proposals and solutions, and develop AI-infused DevOps and platform engineering frameworks.
- Lead cross-functional and client-facing delivery activities, including stakeholder management, RFP support, and proposal evaluation.
Requirements
- BE/B.Tech/MCA and 4–8 years of industry experience.
- Cloud computing experience in multi-cloud Azure and AWS environments, including networking, IAM, architecture, and cost management.
- Hands-on Databricks administration on Azure and/or AWS, including workspace, cluster, and job management.
- Experience setting up CI/CD pipelines with Azure DevOps, GitHub Actions, or GitLab CI.
- Strong understanding of DataOps, MLOps, orchestration, monitoring, governance, and distributed data or ML workloads.
- Hands-on experience with Docker and Kubernetes, including AKS, EKS, or GKE.
- Scripting experience with PowerShell, Python, Groovy, Shell scripting, and cloud-native CLIs.
- Proficiency with infrastructure-as-code tools such as Terraform, ARM Templates, or Bicep and configuration management tools such as Ansible, Chef, or Puppet.
- Experience creating Golden Paths for end-to-end developer workflows and automating applications with Azure DevOps.
- Experience with observability, monitoring, logging, troubleshooting, security, compliance, and platform adoption.
- Preferred certifications include Microsoft Azure DevOps Engineer Expert, Azure infrastructure or solutions architect certifications, AWS DevOps Engineer Professional, AWS Solutions Architect Associate, HashiCorp Terraform Associate, and Certified Kubernetes Administrator.
- Experience with Terraform Cloud, Ansible Tower, regulatory compliance such as GDPR or HIPAA, and recently developed technologies is preferred.
- Strong communication, stakeholder management, leadership, mentoring, analytical, problem-solving, research, presentation, and client-service skills; willingness to travel as required.
Benefits
- Continuous learning opportunities and tools to develop new skills.
- Flexible support to make a meaningful impact and define success individually.
- Transformative leadership through coaching, insights, and development support.
- Diverse and inclusive global culture with collaboration across EY teams and locations.
- The role may require travel according to client and job requirements.
- EY Global Delivery Services operates across Argentina, China, India, the Philippines, Poland, and the UK.
Tech Stack
Categories
Data EngineeringDevOps
About Ernst and Young
Ernst & Young (EY) provides audit/assurance, tax, consulting, strategy and transactions services to enterprises, financial institutions, and public‑sector clients. Structured as a global network of partner‑owned member firms, it sells professional services on a fee basis, including a dedicated Financial Services Organization for banking, insurance, and capital markets. Headquartered in London, EY was formed in 1989 from the merger of Ernst & Whinney and Arthur Young, and operates in 150+ countries.