1 month ago
Kraków, PolandMid Level / Senior
Responsibilities
- Design, develop, and maintain scalable data pipelines and data processing solutions.
- Build Python and Apache Spark workflows for data transformation and processing.
- Work with Hadoop and distributed data-processing environments.
- Build and support cloud-based data solutions on Google Cloud Platform.
- Develop ingestion processes integrating multiple source systems.
- Implement data transformations, validation rules, and data quality checks.
- Troubleshoot pipeline issues and support performance optimization.
- Perform scripting, deployment, and operational activities in Linux environments.
- Collaborate with engineers, architects, analysts, and stakeholders to translate business requirements into technical solutions.
- Participate in code reviews and follow software and data engineering best practices.
- Maintain technical documentation for data flows, dependencies, configurations, and operational procedures.
- Support deployment, testing, stabilization, and ongoing maintenance of data solutions.
Requirements
- 2–4+ years of commercial experience in Data Engineering or a similar role.
- Hands-on programming skills in Python.
- Practical experience building and maintaining Apache Spark data-processing jobs.
- Experience with Hadoop or distributed data-processing ecosystems.
- Good knowledge of Linux and command-line environments.
- Commercial experience with Google Cloud Platform and relevant data services.
- Understanding of ETL/ELT processes, data pipelines, and data transformation concepts.
- Working knowledge of SQL and relational data concepts.
- Understanding of data quality, monitoring, and troubleshooting practices.
- Familiarity with Git and modern software development practices.
- Ability to collaborate effectively with distributed teams in an Agile environment.
- English proficiency at a minimum B2 level.
- Experience with BigQuery, Cloud Storage, Dataproc, Dataflow, Pub/Sub, or Apache Airflow is a plus.
- Financial or banking domain experience, data modeling and data warehouse knowledge, and familiarity with Docker, Kubernetes, and CI/CD are preferred.
Benefits
- Flexible collaboration model based on a B2B contract.
- Hybrid work arrangement, indicated by the #LI-HYBRID designation.
- Opportunity to work on international financial-services projects and AI and automation initiatives.
- Online recruitment process includes recruiter screening, hiring manager technical interview, client interview, and feedback or offer.
Tech Stack
Apache AirflowApache HadoopApache SparkDockerGitGoogle BigQueryGoogle Cloud PlatformKubernetesLinuxPythonSQL
Categories
Data Engineering
About Capco
Capco is a global technology and management consultancy focused on financial services, helping banks, capital markets, insurance, and wealth firms with digital transformation, risk and compliance, and data/AI-enabled modernization. Founded in 1998 and headquartered in London, it operates as a Wipro company. Its business model centers on project-based consulting, managed services, and solution delivery across strategy, engineering, and platforms for regulated institutions.