Data Engineer
Willis Towers Watson2 days ago
Kolkāta, IndiaSenior
Responsibilities
- Build and maintain scalable data pipelines with Azure Databricks, Python, and PySpark.
- Orchestrate data movement and workflows using Azure Data Factory pipelines, triggers, and integration runtimes.
- Ingest and transform large datasets from SQL Server and Oracle.
- Manage data governance and access control with Unity Catalog.
- Implement cloud-native data solutions using Azure or AWS.
- Monitor, troubleshoot, and tune pipelines for performance and reliability.
- Participate in the on-call rotation and support production releases.
- Translate business requirements into robust data solutions.
- Maintain technical documentation and runbooks.
Requirements
- At least five years of experience in data engineering, data warehousing, or related areas.
- Bachelor’s degree in Computer Science, Information Technology, or a related discipline, or equivalent practical experience.
- Strong hands-on experience with Azure Databricks, including jobs, workflows, Delta Lake, and Unity Catalog, in enterprise-scale environments.
- Advanced Python skills and experience developing distributed data processing solutions with PySpark.
- Hands-on experience designing, developing, and optimizing Azure Data Factory pipelines and ETL/ELT processes.
- Strong understanding of Azure cloud services, networking, storage, data modeling, and SQL, preferably T-SQL.
- Working knowledge of Windows and Linux environments.
- Experience with CI/CD and DevOps practices for automated deployment of data solutions, with GitHub Actions preferred.
- Proven ability to troubleshoot complex data processing and pipeline issues in production environments.
Categories
Data Engineering
About Willis Towers Watson
WTW is a global advisory, broking, and software firm serving employers, insurers, and institutional clients across people, risk, and capital. It provides insurance brokerage and risk advisory, employee benefits and pensions consulting, and HR/benefits administration software and analytics. Headquartered in London and listed on NASDAQ (WTW), the company was formed in 2016 via the Willis Group–Towers Watson merger and operates in roughly 140 markets worldwide.