Freelance Senior Databricks Platform Engineer

7 hours ago

Brussels, BelgiumSenior

Responsibilities

Design and build data product platforms and Lakehouse architectures on Databricks using medallion, bronze-silver-gold, Delta Lake, and Unity Catalog patterns.
Build batch and streaming data pipelines with PySpark and Spark SQL, orchestrated through Databricks Workflows and Jobs.
Model data for analytics, business intelligence, and data science use cases, including star schemas and curated datasets.
Ingest data from APIs, files, databases, and SaaS sources into data lakes and Lakehouses.
Implement testable transformations, data quality checks, deduplication, slowly changing dimensions, change data capture, and business rules.
Apply data governance practices including cataloging, lineage, role-based access control, and sensitive-data masking.
Set up CI/CD and deployment pipelines for Databricks assets using Azure DevOps and GitHub.
Automate reusable Infrastructure as Code setup with Terraform.
Monitor, optimize, and troubleshoot pipelines and clusters for performance, robustness, and cost efficiency.
Define coding standards, best practices, and reusable components for data platform teams.

Proven experience with Databricks platform architecture, Unity Catalog, and data governance.
Experience implementing CI/CD and DevOps practices for Databricks using GitHub and enterprise tooling.
Experience with Terraform and Infrastructure as Code implementation and integration.
Experience with Lakehouse architecture and data product platform enablement.
Knowledge of knowledge graphs, ontologies, and agentic AI is a strong plus.

ASAP start date.
Remote work.
Six-month freelance engagement.
Flexible freelance setup with support from the growing adesso Belgium community.
Opportunity to work with an international group and collaborate with cross-functional teams in the international data and AI community.
Rate is to be determined based on location.

11-50 employees