1 month ago
Sofia, BulgariaSenior
Responsibilities
- Design and implement scalable batch, streaming, and near-real-time data platforms and pipelines on AWS with Databricks or Azure/Fabric with Databricks or native Fabric tooling.
- Build ingestion, transformation, curation, lakehouse, and medallion-layer workflows for structured and unstructured data.
- Deliver production-grade datasets and pipelines for analytics, machine learning, causal modeling, optimization, GenAI, RAG, vector-based data flows, and agent-based systems.
- Design logical and physical data models and orchestrate workflows using Airflow, dbt, Databricks Workflows, Azure Data Factory, or equivalent tools.
- Apply architecture, governance, lineage, quality, access-control, and observability practices across data platforms.
- Enable data-serving layers, APIs, feature inputs, and analytical endpoints for downstream ML and AI platforms.
- Monitor and optimize pipelines and infrastructure for performance, scalability, reliability, and cost efficiency.
- Partner with stakeholders to discover, refine, and challenge business requirements and translate them into robust data solutions.
- Collaborate with data scientists, ML engineers, analysts, platform teams, and business stakeholders and contribute to data and AI best practices.
Requirements
- Strong hands-on experience with Apache Spark and Delta Lake, plus strong programming skills in Python and SQL.
- Proven experience building batch and streaming data pipelines and production-grade data platforms.
- Strong understanding of data modeling, data quality, governance, lakehouse architectures, distributed data systems, scalability, reliability, and performance.
- Deep hands-on experience with either AWS data services such as S3, Glue, EMR, Redshift, Kinesis, and Lambda plus Databricks, or Microsoft Azure/Fabric services such as Data Factory, Synapse, Fabric Lakehouse, and Azure Databricks.
- Curiosity, strong problem-solving skills, detailed analysis, experience-based recommendations, and the ability to derive and challenge business requirements.
- Collaborative experience working with cross-functional teams; Agile or consulting experience is beneficial.
- Experience with both AWS and Azure/Fabric, Databricks and native Fabric tooling, GenAI data systems, RAG pipelines, vector databases, LLM data preparation, CI/CD for data pipelines, Terraform, or CloudFormation is a plus.
- Additional experience with Kafka, Spark optimization, causal or experimentation platforms, advanced analytics and ML workloads, data products, or large-scale analytics platforms is valuable.
Benefits
- Hybrid work options and 25 vacation days.
- Co-subsidized transportation and Multisport cards.
- Premium health insurance.
- Training policy covering technical and skills-related events, courses, and certifications.
- Personal career development roadmap guided by performance evaluations.
- Self-care program with psychological consultations and team discussions.
- Cozy office space and team events and company gatherings.
Tech Stack
Amazon RedshiftApache AirflowApache KafkaApache SparkAWSAzureDatabricksdbtGoogle Cloud PlatformMLflowPythonSnowflakeSQLTerraform
Categories
Data Engineering
About Valtech
Valtech provides digital consulting and engineering services to large enterprises, delivering customer experience design, commerce platforms, data/AI solutions, and transformation programs. It works across sectors such as retail, financial services, manufacturing, mobility, health, and travel, offering strategy, design, build, and managed services. Founded in 1993, Valtech is a global company privately held under Kharis Capital.