23 hours ago
Bengaluru, IndiaMid Level / Senior
Responsibilities
- Design, build, and maintain end-to-end data pipelines using Databricks, PySpark, and Spark SQL.
- Implement batch and incremental processing with Delta Lake and multi-hop architecture.
- Develop and optimize Databricks notebooks, jobs, and workflows.
- Ingest, transform, and curate large-scale structured and semi-structured datasets.
- Support analytics, reporting, AI/ML workloads, and downstream data consumption.
- Ensure data quality, reliability, lineage, governance, security, access control, and compliance.
- Optimize Spark jobs for performance and cost efficiency.
- Provide production support and troubleshoot data pipeline issues.
- Document technical designs, data flows, and operational runbooks.
- Mentor junior engineers and contribute to engineering best practices.
- Collaborate with data scientists, analysts, architects, and business teams.
Requirements
- 4–7 years of experience as a Data Engineer with strong Databricks expertise.
- Hands-on experience with Apache Spark, PySpark, and Spark SQL.
- Strong knowledge of Delta Lake and Lakehouse architecture.
- Advanced SQL skills and experience with ETL/ELT patterns and data warehousing concepts.
- Exposure to at least one cloud platform: Azure, AWS, or GCP.
- Understanding of distributed computing concepts.
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field, with a stated 60% or above requirement; Bachelor of Engineering or Bachelor of Technology is required in the education section.
- Experience working in Agile teams.
- Preferred: Unity Catalog and data governance experience, Auto Loader or streaming frameworks, CI/CD for data pipelines, Python for data engineering and automation, and Databricks Associate or Professional certification.
Benefits
- PwC offers inclusive benefits, flexibility programmes, wellbeing support, mentorship, and professional development.
- The role is based in Bangalore, India; travel requirements are not specified.
- The posting states that work visa sponsorship is not available.
Tech Stack
Categories
Data Engineering
About PwC
PwC is a global network of professional services firms providing assurance, tax, and advisory work for businesses and public-sector clients. It sells project-based and managed consulting, audit, and deals services, and implements and integrates business applications. Formed in 1998 by the merger of Price Waterhouse and Coopers & Lybrand, it is headquartered in London and is one of the Big Four accounting firms.