2 months ago
Barcelona, SpainMid Level
Responsibilities
- Operate production ingestion, transformation, and delivery pipelines in Databricks.
- Diagnose and resolve pipeline failures and data quality issues.
- Reverse-engineer and document transformation logic and business rules.
- Migrate legacy tables from Hive Metastore to Unity Catalog.
- Maintain Iceberg-enabled table sharing between Databricks and Snowflake.
- Build and test dbt models, including incremental materializations and data tests.
- Develop and maintain Airflow DAGs, operators, schedules, dependencies, retries, and backfills.
- Validate that migrated Snowflake pipelines produce output equivalent to Databricks versions.
- Contribute to Snowflake modeling, query performance, and compute-cost decisions.
- Work directly with client stakeholders on technical topics.
- Explain production incidents to non-technical stakeholders and provide realistic ETAs.
Requirements
- At least 3 years of experience operating production data pipelines.
- Experience reading, debugging, and modifying PySpark and SQL pipelines.
- Knowledge of Delta Lake MERGE/upsert patterns, table properties, OPTIMIZE, and partitioning.
- Experience with Databricks Workflows, cluster configuration, job troubleshooting, and Unity Catalog.
- Knowledge of Snowflake warehouses, roles, grants, query performance, and compute-cost behavior.
- Experience building dbt models, sources, tests, incremental materializations, and deployment workflows.
- Experience writing and maintaining Airflow DAGs with scheduling, dependency management, retries, backfills, and idempotent task design.
- Strong SQL skills, including window functions, complex joins, and reading transformation logic written by others.
- Proficiency in Python for scripting, automation, and API integration.
- Knowledge of incremental loading, idempotency, late-arriving data, and reprocessing patterns.
- AWS experience including S3 and IAM basics, plus basic working knowledge of Redshift.
- Fluent English for client-facing communication with stakeholders based abroad.
- Ability to work independently in unfamiliar codebases and communicate clearly with technical and non-technical stakeholders.
- Platform migration, open table format, Iceberg, cross-platform sharing, clickstream or web analytics, and undocumented-system stabilization experience are nice-to-have qualifications.
- Ability to use AI tools to improve drafting, analysis, research, process automation, and workflow recommendations is preferred.
Benefits
- Flexible and collaborative culture.
- Professional development opportunities.
- Global opportunities and a vibrant community.
- Total rewards, with specific benefits determined by employment type and location.
- Barcelona hybrid work arrangement.
About Wizeline
Wizeline is a global technology services firm that designs and builds digital products and platforms for enterprises, offering product development, UX, cloud/DevOps, data, and AI services. It works on a consulting and managed-services model to modernize core systems and deliver measurable outcomes. Founded in 2014 and headquartered in San Francisco, it operates internationally and is privately held.