Capgemini

Senior Data Engineer - Databricks

Capgemini
Apply
4 days ago

Base Salary

$84k - $156k/yr

Responsibilities

  • Design, develop, and maintain data pipelines using Databricks, Apache Spark, and Delta Lake.
  • Build and optimize ETL/ELT processes for structured and unstructured data.
  • Develop and maintain PySpark, Scala, and SQL notebooks for data transformation and analysis.
  • Implement Delta Lake bronze, silver, and gold layers using medallion architecture patterns.
  • Optimize Spark jobs through cluster configuration, partitioning, and caching strategies.
  • Implement data quality checks, validation frameworks, and pipeline monitoring.
  • Integrate Databricks with AWS cloud environments.
  • Support automated deployment of notebooks and workflows through CI/CD pipelines.
  • Use Git-based version control, code reviews, and collaborative development practices.
  • Partner with data architects, analysts, and business stakeholders to deliver technical data solutions.

Requirements

  • Bachelor's degree in Computer Science, Information Systems, Engineering, or a related discipline.
  • At least 7 years of experience in data engineering, data warehousing, or big data technologies.
  • At least 3 years of hands-on experience with Databricks and Apache Spark.
  • Strong expertise in PySpark, SQL, and data transformation frameworks.
  • Experience implementing Delta Lake and medallion architecture patterns.
  • Strong understanding of distributed computing and Spark performance optimization.
  • Experience developing and supporting data solutions in AWS Cloud environments.
  • Hands-on experience with CI/CD tools and DevOps practices.
  • Proficiency with Git-based version control systems.
  • Knowledge of data modeling, ETL/ELT design patterns, and modern data lakehouse architectures.
  • Strong communication and stakeholder management skills.

Benefits

  • Paid time off including vacation, company holidays, personal days, and sick leave, with vacation ranging from 12 to 25 days depending on employee grade.
  • Medical, dental, and vision coverage, or provincial healthcare coordination in Canada.
  • Retirement savings plans such as 401(k) in the U.S. and RRSP in Canada.
  • Life and disability insurance.
  • Employee assistance programs and other benefits based on local policy and eligibility.
  • Comprehensive benefits are available to regular, full-time employees and vary by location and eligibility.

Tech Stack

Categories

Data Engineering
Capgemini

About Capgemini

10,000+ employees

Capgemini is a global IT services and consulting firm that delivers strategy, cloud, AI, software engineering, and managed services to large enterprises and public-sector clients. Founded in 1967 and headquartered in Paris, it is publicly traded on Euronext Paris and operates in 50+ countries. The group expanded its engineering capabilities by acquiring Altran in 2020, now operating as Capgemini Engineering.

Contact me