Senior Consultant - Tech Consulting - National - CNS - TC - AI and Data - Noida
Ernst and Young4 days ago
Noida, IndiaSenior
Responsibilities
- Design, develop, and maintain scalable Python-based microservices for data ingestion and processing.
- Build and optimize microservices supporting end-to-end ETL pipelines for large-scale batch and streaming data.
- Develop and manage distributed data-processing workflows using Apache Spark and PySpark.
- Create, schedule, and monitor workflows with Apache Airflow.
- Design and deploy Python data pipelines on AWS EMR or similar big data platforms.
- Implement real-time data ingestion and streaming pipelines using Kafka.
- Ensure data quality, reliability, performance, and scalability of data systems.
- Apply code quality, testing, CI/CD, and production-monitoring best practices.
- Troubleshoot production issues and perform root-cause analysis.
- Collaborate across multiple client departments while meeting commercial and legal requirements.
Requirements
- 6–8 years of experience in data engineering or backend data platform roles.
- At least 4 years of experience in data engineering.
- Strong proficiency in Python and experience building production-grade microservices.
- Hands-on expertise with Apache Spark and PySpark.
- Experience with AWS EMR or equivalent cloud-based big data services and knowledge of the AWS ecosystem.
- Strong experience with Apache Airflow for workflow orchestration.
- Deep understanding of ETL pipeline design and data modeling.
- Hands-on experience with Kafka for inter-domain communication and real-time or streaming data pipelines.
- Strong experience with PostgreSQL or a similar relational database management system.
- Experience working in Agile/Scrum environments.
- Bachelor of Technology in Computer Science and Master of Computer Applications.
- Ability to work collaboratively, solve complex problems practically, and deliver insightful client solutions.
Benefits
- Training, skills development, and learning resources.
- Personalized Career Journey and access to EY career frameworks.
- Opportunities to work with entrepreneurs, game-changers, disruptors, and visionaries across industries.
- Inclusive workplace focused on career development, client service, and employee wellbeing.
- Sector-agnostic national team experience across multiple industries and client departments.
Tech Stack
Categories
Data Engineering
About Ernst and Young
Ernst & Young (EY) provides audit/assurance, tax, consulting, strategy and transactions services to enterprises, financial institutions, and public‑sector clients. Structured as a global network of partner‑owned member firms, it sells professional services on a fee basis, including a dedicated Financial Services Organization for banking, insurance, and capital markets. Headquartered in London, EY was formed in 1989 from the merger of Ernst & Whinney and Arthur Young, and operates in 150+ countries.