DE - Cloud - Senior Data Platform Engineer - GDSN02
Ernst and Young7 days ago
Chennai, IndiaMid Level / Senior
Responsibilities
- Architect and manage Databricks and Snowflake infrastructure across Azure and AWS.
- Establish data governance, access controls, lineage, compliance, security, and network isolation across cloud and on-premises environments.
- Design and implement DataOps and MLOps pipelines for ingestion, transformation, model training, deployment, monitoring, and governance.
- Automate Databricks and Snowflake infrastructure provisioning with Terraform, ARM Templates, or Bicep.
- Integrate data and ML lifecycle workflows with CI/CD tools and source control systems.
- Implement monitoring, logging, alerting, performance optimization, cost optimization, and reliability practices for Databricks workloads.
- Manage secrets, encryption, security scanning, compliance checks, and DevSecOps practices in data pipelines.
- Lead incident response and root cause analysis for platform issues and mentor teams on Databricks, DataOps, and MLOps practices.
- Create Golden Paths for end-to-end developer workflows and contribute to AI-infused DevOps and platform engineering frameworks.
- Support client solution requirements, vendor RFP development, and evaluation of vendor proposals, capabilities, pricing, and support.
- Document architectures and best practices, collaborate with data engineering, analytics, and ML teams, and lead cross-functional platform adoption.
Requirements
- BE, B.Tech, or MCA degree with 4–8 years of industry experience.
- Cloud computing experience in multi-cloud Azure and AWS environments.
- Hands-on Databricks administration on Azure and/or AWS, including workspace, cluster, and job management.
- Experience setting up CI/CD pipelines with Azure DevOps, GitHub Actions, or GitLab CI.
- Strong understanding of DataOps and MLOps practices, including orchestration, monitoring, and governance.
- Experience with Docker and Kubernetes, including AKS, EKS, or GKE.
- Scripting knowledge in PowerShell, Python, Groovy, and shell scripting, plus cloud-native command-line interfaces.
- Experience with infrastructure as code using Terraform, ARM Templates, or Bicep.
- Experience with configuration management tools such as Ansible, Chef, or Puppet.
- Experience creating Golden Paths and automating application and infrastructure workflows.
- Strong cloud architecture, monitoring, logging, troubleshooting, security, compliance, communication, stakeholder management, and leadership skills.
- Preferred certifications include Microsoft Azure DevOps Engineer Expert, Azure infrastructure or solutions architect certifications, AWS DevOps Engineer Professional, AWS Solutions Architect Associate, HashiCorp Terraform Associate, and Certified Kubernetes Administrator.
- Experience with Terraform Cloud, Ansible Tower, regulatory compliance, and recently developed technologies is desirable.
- Excellent written and oral communication skills; publishing and conference-level presentation experience is a plus.
- Ability to travel in accordance with client and job requirements.
Benefits
- Continuous learning opportunities and tools to develop new skills.
- Flexible support to define and achieve career success and make a meaningful impact.
- Transformative leadership through coaching, insights, and confidence-building.
- Diverse and inclusive culture with opportunities to collaborate across EY teams and global locations.
- Global Delivery Services collaboration across Argentina, China, India, the Philippines, Poland, and the UK.
- Travel may be required in accordance with client and job requirements.
Tech Stack
Categories
Data EngineeringDevOps
About Ernst and Young
Ernst & Young (EY) provides audit/assurance, tax, consulting, strategy and transactions services to enterprises, financial institutions, and public‑sector clients. Structured as a global network of partner‑owned member firms, it sells professional services on a fee basis, including a dedicated Financial Services Organization for banking, insurance, and capital markets. Headquartered in London, EY was formed in 1989 from the merger of Ernst & Whinney and Arthur Young, and operates in 150+ countries.