4 days ago
Pune, IndiaStaff+
Responsibilities
- Design and develop scalable data solutions and products aligned with business requirements and modern software engineering practices.
- Own logical and physical data designs for curated, conformed, and consumption-ready datasets.
- Translate business and analytical requirements into dimensional, domain-oriented, and lakehouse data models.
- Build and operate ingestion, transformation, curation, batch, and streaming pipelines using AWS, Apache Spark, and Databricks.
- Develop and manage data workflows with Apache Airflow and Astronomer, including scheduling, monitoring, and alerting.
- Implement CI/CD, automated testing, deployment controls, data quality checks, reconciliation controls, metadata, and lineage.
- Ensure solutions meet cloud, security, privacy, resilience, regulatory, audit, risk, and governance requirements.
- Assess architecture impacts on risk, capacity, performance, and cost, including project change and ongoing run costs.
- Support production releases, fault finding, incident analysis, performance optimization, and continuous improvement.
- Provide technical leadership, mentor junior engineers, influence stakeholders, and promote engineering excellence and automation.
Requirements
- Strong experience in AWS data engineering, including S3, Glue, EMR, Athena, and IAM.
- Advanced Python development skills for data engineering.
- Strong hands-on experience with Apache Spark and PySpark.
- Proven Databricks experience, including development, optimization, notebooks, and jobs.
- Hands-on experience with Apache Airflow and Astronomer.
- Strong SQL skills and experience working with large datasets.
- Experience with Git-based version control, CI/CD, and DevOps practices for data platforms.
- Solid understanding of data modeling, lakehouse architecture, and analytics-ready data design.
- Experience implementing data quality, validation, and reconciliation frameworks.
- Familiarity with metadata, lineage, and data governance concepts.
- Experience in financial services or regulated environments is preferred.
- Understanding of risk, controls, resilience, security, audit, and other non-functional requirements.
- Demonstrated ownership and accountability for technical deliverables, strong problem-solving, communication, and analytical skills.
- Experience mentoring junior engineers and promoting engineering best practices, automation, and continuous improvement.
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
- AWS or Databricks certifications are desirable.
Tech Stack
Categories
Data Engineering
About Barclays
Barclays is a UK‑based universal bank that provides retail and commercial banking, credit cards (Barclaycard), wealth management, and investment banking to consumers, businesses, and institutions. Publicly listed in London and New York, it generates revenue from net interest income, fees, and markets activities, and operates across Europe, the Americas, the Middle East, and Asia. Founded in 1690 and headquartered in London, it serves customers through digital channels and a branch network.