1 month ago
Remote, North MacedoniaSenior
Responsibilities
- Design and implement scalable data platforms and batch, streaming, and near-real-time pipelines on AWS or Azure/Fabric using Databricks or native Fabric tooling.
- Build ingestion, transformation, curation, lakehouse, and medallion-layer workflows for structured and unstructured data.
- Deliver production-grade datasets and pipelines for analytics, machine learning, causal modeling, optimization, GenAI, RAG, vector-based data flows, and agent-based systems.
- Design logical and physical data models and orchestrate workflows with Airflow, dbt, Databricks Workflows, Azure Data Factory, or equivalent tools.
- Implement data governance, lineage, quality, access control, observability, monitoring, and SLA adherence using platform-appropriate tools.
- Enable data-serving layers, APIs, feature inputs, and analytical endpoints for downstream ML and AI platforms.
- Monitor and optimize pipeline and infrastructure performance, scalability, reliability, and cost across cloud ecosystems.
- Collaborate with data scientists, ML engineers, analysts, platform teams, and business stakeholders to derive and refine requirements and support adoption of data products.
Requirements
- Strong hands-on experience with Apache Spark, Delta Lake, Python, and SQL.
- Proven experience building batch and streaming data pipelines and production-grade data platforms.
- Strong understanding of data modeling, data quality, governance, lakehouse architectures, distributed data systems, scalability, reliability, and performance.
- Deep hands-on experience in either AWS data services and Databricks Lakehouse Platform or Microsoft Azure/Fabric and Azure Databricks.
- AWS-track experience may include S3, Glue, EMR, Redshift, Kinesis, and Lambda; Azure/Fabric-track experience may include Data Factory, Synapse, Fabric Lakehouse, and Azure Databricks.
- Strong problem-solving, collaboration, detailed analysis, and business-requirements discovery skills.
- Experience in Agile or consulting environments is beneficial.
- Experience with both AWS and Azure/Fabric, GenAI data systems, RAG pipelines, vector databases, LLM data preparation, CI/CD for data pipelines, Terraform, CloudFormation, Kafka, Spark optimization, causal or experimentation platforms, data products, or large-scale analytics platforms is advantageous.
Benefits
- Private health insurance for employees.
- Education program with training and certification opportunities.
- Wellbeing program.
- Free coffee, drinks, snacks, and company dinners.
- Company social events including trips, activities, tastings, picnics, and cooking classes.
- Competitive salary and 24 days of vacation.
- Challenging projects, collaborative colleagues, and an open feedback culture.
Tech Stack
Amazon RedshiftApache AirflowApache KafkaApache SparkAWSAzureDatabricksdbtGoogle Cloud PlatformMLflowPythonSnowflakeSQLTerraform
Categories
Data Engineering
About Valtech
Valtech provides digital consulting and engineering services to large enterprises, delivering customer experience design, commerce platforms, data/AI solutions, and transformation programs. It works across sectors such as retail, financial services, manufacturing, mobility, health, and travel, offering strategy, design, build, and managed services. Founded in 1993, Valtech is a global company privately held under Kharis Capital.