7 days ago
Bengaluru, IndiaMid Level
Responsibilities
- Design and build Azure Databricks pipelines that ingest, transform, and deliver data for analytics and reporting.
- Develop cloud data engineering solutions across Azure data services, data lakes, and modern data warehouses.
- Optimize pipeline performance, query execution, data storage, and scalability.
- Implement data quality checks, validation, monitoring, and batch or streaming workflows.
- Translate business requirements into data models, integration patterns, and reusable engineering components.
- Support legacy data modernization through cloud migration, consolidation, and platform refactoring.
- Use Apache Airflow, Azure Data Factory, and related orchestration tools to manage workflows.
- Review code, troubleshoot pipeline issues, guide assigned workstreams, and mentor junior team members.
- Create design documents, unit test plans, code review reports, architecture documentation, user manuals, and operations handoff guides.
Requirements
- Bachelor’s degree or higher.
- At least 4 years of experience with Azure ADLS, Databricks, Data Flows, HDInsight, and Azure Analysis Services.
- Knowledge of data management, data modeling, big data frameworks, and data warehouse lifecycles.
- Mandatory experience with Hadoop and Spark.
- Experience extracting data from flat files, XML, JSON, Parquet, big data appliances, and relational database management systems.
- Strong SQL skills, including joins, groups, functions, and stored procedures.
- Experience creating and verifying unit and integration tests.
- Experience with data migration processes, methods, and project lifecycles.
- Good Unix shell scripting skills and strong analytical and troubleshooting abilities.
- Knowledge of quality processes and implementation, application DevOps tools, Git, and CI/CD frameworks.
- Experience with Jenkins or GitLab and source code management tools such as CodePipeline, CodeBuild, and CodeCommit.
- Oral and written proficiency in English.
- Preferred fields of study include Management Information Systems, Computer and Information Science, Systems Engineering, engineering disciplines, Mathematics, Statistics, or Mathematical Statistics.
- Preferred experience includes stream processing with Storm or Spark Streaming, big data ML toolkits such as Mahout, SparkML, or H2O, Azure Functions, Python, Snowflake, Apache Airflow, and Azure Data Factory.
- Azure or Databricks certifications are desirable.
Benefits
- Work with global teams across Advisory, Assurance, Tax, and Business Services in PwC’s Acceleration Center India.
- Receive hands-on learning opportunities and work with modern cloud and data engineering tools.
- The posting does not specify a travel requirement.
- The posting does not state a work arrangement, contract duration, or compensation details.
Tech Stack
Apache AirflowApache HadoopApache SparkApache StormAzureDatabricksGitH2O.aiJenkinsPythonSnowflakeSQL
Categories
Data Engineering
About PwC
PwC is a global network of professional services firms providing assurance, tax, and advisory work for businesses and public-sector clients. It sells project-based and managed consulting, audit, and deals services, and implements and integrates business applications. Formed in 1998 by the merger of Price Waterhouse and Coopers & Lybrand, it is headquartered in London and is one of the Big Four accounting firms.