PwC

IN_Senior Associate_Databricks Data Engineer_GCC_Advisory_Bangalore

PwC
Apply
23 hours ago
Bengaluru, IndiaMid Level / Senior

Responsibilities

  • Design, build, and maintain end-to-end data pipelines using Databricks, PySpark, and Spark SQL.
  • Implement batch and incremental processing with Delta Lake and multi-hop architecture.
  • Develop and optimize Databricks notebooks, jobs, and workflows.
  • Ingest, transform, and curate large-scale structured and semi-structured datasets.
  • Support analytics, reporting, downstream data consumption, and AI/ML workloads.
  • Ensure data quality, reliability, lineage, governance, security, access control, and compliance.
  • Optimize Spark jobs for performance and cost efficiency.
  • Provide production support and troubleshoot data pipeline issues.
  • Document technical designs, data flows, and operational runbooks.
  • Mentor junior engineers and contribute to engineering best practices.

Requirements

  • 4–7 years of experience as a Data Engineer with strong Databricks expertise.
  • Hands-on experience with Apache Spark, PySpark, and Spark SQL.
  • Strong knowledge of Delta Lake and Lakehouse architecture.
  • Advanced SQL skills and experience with ETL/ELT patterns and data warehousing concepts.
  • Exposure to at least one cloud platform: Azure, AWS, or GCP.
  • Understanding of distributed computing concepts and experience working in Agile teams.
  • Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field, with the posting specifying Bachelor of Technology or Bachelor of Engineering and a 60% or higher result.
  • Preferred qualifications include Unity Catalog and data governance experience, Auto Loader or streaming framework exposure, CI/CD for data pipelines, Python for data engineering and automation, and Databricks certification.
  • Experience with Snowflake is listed as a required skill; Alteryx and additional data analytics capabilities are listed among optional skills.

Benefits

  • Inclusive benefits, flexibility programs, wellbeing support, mentorship, and opportunities for learning and professional growth are offered.
  • The role is based in Bangalore, with travel requirements not specified.
  • The posting states that work visa sponsorship is not available.

Categories

Data Engineering
PwC

About PwC

10,000+ employees

PwC is a global network of professional services firms providing assurance, tax, and advisory work for businesses and public-sector clients. It sells project-based and managed consulting, audit, and deals services, and implements and integrates business applications. Formed in 1998 by the merger of Price Waterhouse and Coopers & Lybrand, it is headquartered in London and is one of the Big Four accounting firms.

Contact me