Senior Associate/Manager Data Engineer Consultant (Data Bricks) / Technology Consulting
Ernst and Young1 month ago
Singapore, SingaporeSenior
Responsibilities
- Design and implement enterprise-scale data lakes, warehouses, and real-time streaming architectures.
- Develop and maintain ETL/ELT pipelines for structured and unstructured data from diverse sources.
- Build Databricks notebooks and PySpark, Python, and SQL solutions for ETL automation and data transformation.
- Implement data manipulation, validation, error handling, governance, security, and quality controls.
- Monitor and optimize pipeline performance, scalability, and cost-effectiveness.
- Test, debug, troubleshoot, and verify data integrity across data pipeline stages.
- Provide technical mentorship to junior data engineers and analysts and lead code reviews.
- Collaborate with data scientists, analysts, software engineers, and technical and business stakeholders.
- Support ETL migration projects, technical design sessions, documentation, and data infrastructure planning.
Requirements
- Bachelor’s degree in Computer Science, Engineering, Data Science, or a related field is required; a master’s degree in Data Engineering, Computer Science, or an MBA is preferred.
- 5–7 years of data engineering experience with AWS-native implementations.
- Experience contributing to ETL migration projects and developing production data pipelines.
- Hands-on experience with Databricks notebooks, PySpark, Python, and SQL for ETL automation.
- Experience with large-scale data extraction, transformation, loading, validation, debugging, and optimization.
- Proficiency with complex SQL joins, aggregations, and database operations in Databricks environments.
- Experience with AWS data services, data warehouse design, real-time streaming, data exchange platforms, and API integration.
- Ability to collaborate across technical and business teams, communicate technical concepts, and manage stakeholders.
- Technical mentorship, coaching, problem-solving, decision-making, documentation, and organizational skills.
- Government-sector data and compliance experience, HR and workforce analytics experience, mentorship experience, and relevant AWS, Databricks, Amazon Redshift, Snowflake, or Agile/Scrum certifications are preferred.
Benefits
- Flexible working arrangements are available.
- Continuous learning, coaching, and career development opportunities are provided.
- EY offers an inclusive and diverse culture with flexibility to make a meaningful impact.
- The role includes a competitive remuneration package tied to individual and team performance.
Tech Stack
Categories
Data Engineering
About Ernst and Young
Ernst & Young (EY) provides audit/assurance, tax, consulting, strategy and transactions services to enterprises, financial institutions, and public‑sector clients. Structured as a global network of partner‑owned member firms, it sells professional services on a fee basis, including a dedicated Financial Services Organization for banking, insurance, and capital markets. Headquartered in London, EY was formed in 1989 from the merger of Ernst & Whinney and Arthur Young, and operates in 150+ countries.