20 hours ago
Bengaluru, IndiaMid Level / Senior
Responsibilities
- Design, develop, and maintain scalable data pipelines using Apache Spark and Scala.
- Build and optimize ETL/ELT processes for large-scale structured and unstructured datasets.
- Develop data processing and automation solutions using Python and Java.
- Design data models, transformation logic, and data integration workflows.
- Work with MongoDB, relational databases, and data warehouses.
- Write optimized SQL queries and stored procedures and perform performance tuning.
- Collaborate with data scientists, analysts, and business stakeholders to deliver data solutions.
- Implement data quality, validation, monitoring, and governance controls.
- Troubleshoot data pipeline failures and performance bottlenecks.
- Build and maintain CI/CD pipelines and deployment processes for data engineering solutions.
- Contribute to solution architecture, technical design discussions, and technology decisions.
- Support production systems and resolve operational issues.
- Mentor junior engineers and share knowledge across the team.
Requirements
- Expertise in Apache Spark, Scala, Python, SQL, MongoDB, and Java.
- Experience developing scalable data processing solutions and ETL/ELT processes.
- Ability to work with structured and unstructured datasets, data models, relational databases, and data warehouses.
- Ability to collaborate with data scientists, analysts, and business stakeholders.
- Knowledge of data quality, validation, monitoring, governance, coding standards, security policies, and best practices.
Tech Stack
Categories
Data Engineering
About Allstate
Allstate is a major U.S. personal-lines insurer that sells auto, homeowners, renters, and life policies to consumers through agents and direct online channels. It earns premiums and service fees across insurance and protection products, including telematics-based programs and device/identity protection. Founded in 1931 and headquartered in Northbrook, Illinois, Allstate is publicly traded on the NYSE under the ticker ALL.