Senior Data Engineer-1
Prudential plc10 days ago
Bengaluru, IndiaSenior
Responsibilities
- Design, build, test, deploy, and maintain batch and near-real-time data pipelines on the Unified Data Platform.
- Develop data ingestion, transformation, enrichment, reconciliation, and publishing processes.
- Build and maintain commissions-calculation data processing covering rules, rates, adjustments, incentives, deductions, and outputs.
- Implement reusable data models and curated datasets aligned with enterprise and platform standards.
- Ensure data accuracy, completeness, lineage, traceability, security, and auditability across pipelines and calculation processes.
- Implement automated data-quality checks, controls, alerts, and exception handling.
- Optimize pipeline performance, reliability, scalability, and cost.
- Use source control, automated testing, CI/CD, environment controls, and deployment standards.
- Monitor production pipelines, investigate failures and data discrepancies, and provide operational support.
- Enhance and maintain core platform capabilities, engineering standards, and reusable components.
Requirements
- At least 5 years of experience in data engineering or software engineering.
- Strong hands-on experience with Python, SQL, Databricks, Spark, and data pipeline development.
- Experience designing ETL/ELT solutions, data models, transformations, and workflow orchestration.
- Experience implementing complex calculation logic, reconciliation, and financial or commissions-related data processing.
- Understanding of data quality, observability, lineage, metadata, data contracts, and access controls.
- Experience with Delta Lake, Unity Catalog, Git, automated testing, CI/CD, and cloud data platforms.
- Understanding of APIs, file-based integration, event processing, and secure data exchange.
- Knowledge of software engineering practices, production support, and Agile delivery.
- Strong problem analysis, troubleshooting, communication, and collaboration skills.
- Bachelor’s degree in Computer Science, Data Engineering, Information Systems, Engineering, or an equivalent qualification.
- Fluent written and spoken English.
- Insurance or financial services experience is advantageous.
Tech Stack
Categories
Data Engineering
About Prudential plc
Prudential plc provides life and health insurance and asset management for consumers across Asia and Africa, including through its Eastspring Investments arm. The public company is headquartered in Hong Kong with dual primary listings in Hong Kong and London, and traces its origins to 1848 in the UK. Its products span protection, savings, and retirement, distributed via agents, bancassurance partners, and digital channels.