1 month ago
Remote, PortugalSenior
Responsibilities
- Design and implement scalable cloud data platforms and batch, streaming, and near-real-time pipelines on AWS/Databricks or Azure/Fabric.
- Develop ingestion, transformation, curation, data modeling, and data-serving workflows for structured and unstructured data.
- Implement lakehouse architectures and bronze, silver, and gold medallion layers.
- Deliver production-grade datasets and pipelines supporting analytics, machine learning, causal modeling, optimization, GenAI, RAG, LLM, and agent-based systems.
- Orchestrate and automate end-to-end workflows while maintaining reliability, scalability, performance, and maintainability.
- Apply data governance, lineage, quality, access-control, monitoring, observability, and SLA practices.
- Optimize pipelines and infrastructure for performance, scalability, cost efficiency, and production readiness.
- Discover and refine business requirements by analyzing stakeholder needs, challenging assumptions, and recommending better alternatives.
- Collaborate with data scientists, ML engineers, analysts, platform teams, and business stakeholders to deliver robust data products.
Requirements
- Strong hands-on experience with Apache Spark and Delta Lake.
- Strong programming skills in Python and SQL.
- Proven experience building batch and streaming data pipelines and production-grade data platforms.
- Solid understanding of data modeling, data quality, governance, lakehouse architectures, distributed data systems, scalability, reliability, and performance.
- Deep hands-on expertise in either AWS data services and Databricks Lakehouse Platform or Microsoft Azure/Fabric and associated data tooling.
- AWS track experience may include S3, Glue, EMR, Redshift, Kinesis, and Lambda.
- Azure/Fabric track experience may include Data Factory, Synapse, Fabric Lakehouse, and Azure Databricks.
- Strong problem-solving, collaboration, detailed analysis, business curiosity, and experience-based requirements discovery skills.
- Experience in both AWS and Azure/Fabric, Snowflake, GCP, GenAI data systems, RAG pipelines, vector databases, LLM data preparation, CI/CD for data pipelines, Terraform, CloudFormation, Kafka, Spark optimization, advanced analytics, causal or experimentation platforms, data products, or large-scale analytics platforms is beneficial.
Benefits
- Remote and hybrid work options are available depending on country.
- Career advancement, international mobility, professional development programs, training, and access to industry experts are offered.
- The position is based in Portugal, with location-specific benefits provided during the hiring process.
Tech Stack
Amazon RedshiftApache AirflowApache KafkaApache SparkAWSAzureDatabricksdbtGoogle Cloud PlatformMLflowPythonSnowflakeSQLTerraform
Categories
Data Engineering
About Valtech
Valtech provides digital consulting and engineering services to large enterprises, delivering customer experience design, commerce platforms, data/AI solutions, and transformation programs. It works across sectors such as retail, financial services, manufacturing, mobility, health, and travel, offering strategy, design, build, and managed services. Founded in 1993, Valtech is a global company privately held under Kharis Capital.