EY - GDS Consulting - AI And DATA -AI Data Platform Engineer - AWS - Senior
Ernst and Young15 days ago
Hyderābād, IndiaSenior
Responsibilities
- Design and build production-grade AWS data pipelines using Glue, S3, PySpark, Athena, Redshift, EMR, MWAA/Airflow, Step Functions, Lambda, EventBridge, CloudWatch, IAM, and KMS.
- Develop reusable ingestion frameworks for batch, streaming, event-driven, CDC, API-based, file-based, database, and third-party service integration patterns.
- Build and optimize lakehouse data layers, partitioning, metadata management, file formats, and Spark/Glue/EMR workloads for scalability, performance, cost efficiency, and operational stability.
- Create AWS platform accelerators for onboarding, pipeline templates, orchestration, monitoring, reconciliation, deployment, logging, and support runbooks.
- Implement Git-based delivery, branching, pull requests, code reviews, CI/CD, infrastructure as code, controlled releases, and environment promotion.
- Integrate AWS data platforms with enterprise APIs, source applications, messaging and event services, governance tools, security platforms, and analytics consumers.
- Integrate data platforms with SageMaker for data preparation, feature engineering, model training, deployment, MLOps workflows, and inference-ready data products.
- Support SageMaker Pipelines, Feature Store, Model Registry, Model Monitor, Bedrock where relevant, vector stores, semantic search, and RAG-ready data products.
- Implement data security, governance, observability, alerting, retry logic, restartability, root-cause analysis, incident response, SLA/SLO tracking, and production reliability management.
- Collaborate with infrastructure, IAM, network, DBA, application, support, architecture, security, and SRE teams, and mentor engineers.
Requirements
- 5–10 years of experience in data engineering, data platform operations, analytics engineering, platform engineering, or AI platform enablement.
- Strong hands-on implementation experience with cloud data platforms, APIs, Git connectivity, CI/CD, governed access patterns, SRE practices, and production operations.
- Hands-on experience with AWS data platforms, including Glue, S3, Athena, Redshift, EMR, MWAA/Airflow, Step Functions, Lambda, EventBridge, and CloudWatch.
- Experience with Python, PySpark, SQL, APIs, Git, shell scripting, unit testing, integration testing, and data pipeline testing.
- Experience with Terraform/OpenTofu, CloudFormation, GitHub Actions, Azure DevOps, Jenkins, Docker, Kubernetes/EKS, and policy-as-code.
- Knowledge of SageMaker, SageMaker Pipelines, Feature Store, Model Registry, Model Monitor, Bedrock, agentic AI, and RAG-ready data products.
- Experience with IAM, KMS, Lake Formation, Glue Data Catalog, Macie, Immuta, Purview, CloudTrail, data quality, observability, Data SRE, and FinOps.
- Strong engineering execution with architecture awareness, delivery ownership, reusable framework design, secure implementation patterns, operational controls, and production-ready services.
- Ability to mentor engineers, collaborate with architects and security/SRE teams, and adopt AI-native and agentic engineering methods.
- Relevant cloud/platform, data engineering, DevOps, security, governance, or AI/ML engineering certifications are preferred.
Benefits
- Support, coaching, and feedback from colleagues.
- Opportunities to develop new skills and progress your career.
- Individual progression planning, education, coaching, and practical development experience.
- Interdisciplinary work emphasizing quality and knowledge exchange.
- Freedom and flexibility to handle the role in a way that is right for the employee.
Tech Stack
Categories
Data EngineeringDevOps
About Ernst and Young
Ernst & Young (EY) provides audit/assurance, tax, consulting, strategy and transactions services to enterprises, financial institutions, and public‑sector clients. Structured as a global network of partner‑owned member firms, it sells professional services on a fee basis, including a dedicated Financial Services Organization for banking, insurance, and capital markets. Headquartered in London, EY was formed in 1989 from the merger of Ernst & Whinney and Arthur Young, and operates in 150+ countries.