4 days ago
Base Salary
$84k - $156k/yr
Responsibilities
- Design, develop, and maintain data pipelines using Databricks, Apache Spark, and Delta Lake.
- Build and optimize ETL/ELT processes for structured and unstructured data.
- Develop and maintain PySpark, Scala, and SQL notebooks for data transformation and analysis.
- Implement Delta Lake bronze, silver, and gold layers using medallion architecture patterns.
- Optimize Spark jobs through cluster configuration, partitioning, and caching strategies.
- Implement data quality checks, validation frameworks, and pipeline monitoring.
- Integrate Databricks with AWS cloud environments.
- Support automated deployment of notebooks and workflows through CI/CD pipelines.
- Use Git-based version control, code reviews, and collaborative development practices.
- Partner with data architects, analysts, and business stakeholders to deliver technical data solutions.
Requirements
- Bachelor's degree in Computer Science, Information Systems, Engineering, or a related discipline.
- At least 7 years of experience in data engineering, data warehousing, or big data technologies.
- At least 3 years of hands-on experience with Databricks and Apache Spark.
- Strong expertise in PySpark, SQL, and data transformation frameworks.
- Experience implementing Delta Lake and medallion architecture patterns.
- Strong understanding of distributed computing and Spark performance optimization.
- Experience developing and supporting data solutions in AWS Cloud environments.
- Hands-on experience with CI/CD tools and DevOps practices.
- Proficiency with Git-based version control systems.
- Knowledge of data modeling, ETL/ELT design patterns, and modern data lakehouse architectures.
- Strong communication and stakeholder management skills.
Benefits
- Paid time off including vacation, company holidays, personal days, and sick leave, with vacation ranging from 12 to 25 days depending on employee grade.
- Medical, dental, and vision coverage, or provincial healthcare coordination in Canada.
- Retirement savings plans such as 401(k) in the U.S. and RRSP in Canada.
- Life and disability insurance.
- Employee assistance programs and other benefits based on local policy and eligibility.
- Comprehensive benefits are available to regular, full-time employees and vary by location and eligibility.
Tech Stack
Categories
Data Engineering
About Capgemini
Capgemini is a global IT services and consulting firm that delivers strategy, cloud, AI, software engineering, and managed services to large enterprises and public-sector clients. Founded in 1967 and headquartered in Paris, it is publicly traded on Euronext Paris and operates in 50+ countries. The group expanded its engineering capabilities by acquiring Altran in 2020, now operating as Capgemini Engineering.
