Lead Engineer– Data Platform
D.B. Group2 months ago
Madrid, SpainStaff+
Responsibilities
- Own the architecture, design, and evolution of enterprise data platform capabilities across hybrid cloud and on-premises environments.
- Define technical roadmaps, engineering standards, and reusable patterns for orchestration, scheduling, dependency management, and data processing.
- Lead architectural decisions, proofs of concept, adoption plans, and platform automation using infrastructure as code, CI/CD, automated testing, and self-service capabilities.
- Establish reliability, observability, monitoring, alerting, failure recovery, capacity management, and service-level objective standards.
- Integrate batch and streaming pipelines across Snowflake, Airflow, and Kafka.
- Partner with architecture, security, governance, and application teams to meet enterprise and regulatory requirements.
- Remain hands-on in architecture and code, conduct design and code reviews, mentor engineers, and communicate with technical and business stakeholders.
Requirements
- 12+ years of experience in data or platform engineering, including designing and operating enterprise-scale data platforms.
- Deep hands-on expertise with Snowflake across architecture, ingestion, transformation, workload management, performance, cost optimization, secure data sharing, and governance.
- Strong production experience with Apache Airflow, including reusable DAG patterns, dependency management, scheduling, testing, monitoring, and failure recovery.
- Strong expertise in Apache Kafka and event-driven architectures, including topic design, producers and consumers, schema management, Kafka Connect, delivery semantics, and replay strategies.
- Proven ability to integrate batch and streaming pipelines across Snowflake, Airflow, and Kafka.
- Advanced Python and SQL skills; Java experience for Kafka-based services and connectors is beneficial.
- Strong experience with Terraform, Git, CI/CD, and automated testing; OpenShift or Kubernetes experience is desirable.
- Strong understanding of reliability, observability, data quality, security, access management, and compliance in regulated environments.
- Proven technical leadership in setting engineering direction, reviewing designs and code, mentoring engineers, and communicating with technical and business stakeholders.
- Fluent written and spoken English, strong problem-solving and strategic-thinking skills, and the ability to use AI tools responsibly and critically.
- A relevant degree is advantageous.
Benefits
- Personalized benefits plan including healthcare, company perks, and retirement-related benefits.
- Hybrid working model balancing office collaboration with working from home; the specific split varies by business group and is discussed during the application and interview process.
- Interview adjustments and assistance are available for disabilities or long-term health conditions.
Tech Stack
Categories
Data Engineering
About D.B. Group
D.B. Group is an Italian freight forwarder and logistics provider offering air, ocean, road, and rail transport, customs brokerage, and supply chain management for global shippers. Founded in 1980 and headquartered in Montebelluna, Veneto, it is privately held and operates through 79 subsidiaries with a worldwide partner network. The company delivers end-to-end forwarding and integrated logistics services, from consolidation and warehousing to door-to-door deliveries and trade compliance.