DE - Cloud - Senior Data Platform Engineer - GDSN02
Ernst and Young23 days ago
Chennai, IndiaMid Level / Senior
Responsibilities
- Architect and manage Databricks and Snowflake infrastructure across Azure, AWS, and on-premises environments.
- Establish data governance frameworks covering access controls, lineage, compliance, and multi-cloud data architectures.
- Design and implement DataOps and MLOps pipelines for ingestion, transformation, model training, deployment, monitoring, orchestration, and versioning.
- Automate infrastructure provisioning for Databricks and Snowflake using Terraform, ARM Templates, or Bicep.
- Integrate data, code, and model lifecycles with Azure DevOps, GitHub Actions, Jenkins, and other CI/CD platforms.
- Implement monitoring, logging, alerting, incident response, root-cause analysis, performance optimization, and reliability practices for Databricks workloads.
- Apply security, DevSecOps, secrets management, network isolation, encryption, compliance checks, and security scanning across pipelines.
- Collaborate with data engineering, analytics, ML, and client teams to automate workflows and drive platform adoption.
- Document architecture and operational practices, mentor teams, and lead cross-functional initiatives.
- Evaluate software packages and vendor proposals, develop RFPs, and assess business and technology fit, pricing, and support.
- Design and develop AI-infused DevOps and platform engineering frameworks.
- Participate in people initiatives and travel according to client and job requirements.
Requirements
- BE/B.Tech/MCA with 4–8 years of industry experience.
- Knowledge of cloud computing and architecture in multi-cloud environments, especially Azure and AWS.
- Hands-on Databricks administration on Azure and/or AWS, including workspace, cluster, and job management.
- Experience setting up CI/CD pipelines with Azure DevOps, GitHub Actions, GitLab CI, or comparable tools.
- Strong understanding of DataOps, MLOps, orchestration, monitoring, governance, and distributed data and ML workloads.
- Hands-on containerization experience with Docker and Kubernetes, including AKS, EKS, or GKE.
- Scripting experience with PowerShell, Python, Groovy, shell scripting, and cloud-native CLIs.
- Proficiency with infrastructure-as-code tools such as Terraform, ARM Templates, or Bicep.
- Experience with configuration management tools such as Ansible, Chef, or Puppet.
- Experience creating Golden Paths for end-to-end developer workflows and automating applications and infrastructure.
- Experience with cloud networking, IAM, cost management, security, compliance, monitoring, logging, troubleshooting, and observability.
- Strong communication, stakeholder management, analytical, problem-solving, mentoring, and cross-functional leadership skills.
- Preferred certifications include Microsoft Azure DevOps Engineer Expert, Azure Infrastructure Solutions, Azure Solutions Architect Expert, AWS DevOps Engineer Professional, AWS Solutions Architect Associate, HashiCorp Terraform Associate, and Certified Kubernetes Administrator.
- Preferred experience includes Terraform Cloud, Ansible Tower, GitHub Actions, and GitHub Copilot.
- Experience with regulatory compliance such as GDPR and HIPAA is a plus.
- Experience designing and conducting research and experiments with recently developed technologies.
Benefits
- Continuous learning opportunities and access to tools, coaching, skills, and insights.
- Flexible support to define and achieve success in a personally meaningful way.
- Transformative leadership development, coaching, and leadership insights.
- Diverse and inclusive culture with opportunities to use your voice and support others.
- Global collaboration across EY teams, service lines, geographies, and sectors.
- Travel may be required according to client and job requirements.
Tech Stack
Categories
Data EngineeringDevOps
About Ernst and Young
Ernst & Young (EY) provides audit/assurance, tax, consulting, strategy and transactions services to enterprises, financial institutions, and public‑sector clients. Structured as a global network of partner‑owned member firms, it sells professional services on a fee basis, including a dedicated Financial Services Organization for banking, insurance, and capital markets. Headquartered in London, EY was formed in 1989 from the merger of Ernst & Whinney and Arthur Young, and operates in 150+ countries.