Data Engineer (m/f/d)
Schwarz Group3 months ago
Barcelona, SpainMid Level
Responsibilities
- Design, build, and maintain batch and real-time data pipelines.
- Conduct data operations including exploration, provisioning, cleaning, testing, and ALM.
- Develop and implement statistical and machine learning algorithms.
- Collaborate with business stakeholders to translate requirements into technical solutions.
- Ensure data integrity, quality, and availability across all systems.
- Extract actionable business insights.
Requirements
- Proficiency in Python and SQL.
- Experience with Big Data frameworks such as Spark, Kafka, Kubernetes, and Hive.
- Familiarity with relational and non-relational databases, including SQL Server, MongoDB, ELK, and Cassandra.
- Knowledge of machine learning tools including Scikit-Learn, Pandas, Spark ML, and MLflow is preferred.
- Knowledge of infrastructure deployment and DevOps is preferred.
- Experience with .NET is preferred.
- Agile methodology experience.
- Strong problem-solving, adaptability, communication, collaboration, and continuous-learning skills.
- Fluency in English and comfort working in a multicultural environment.
Tech Stack
Apache CassandraApache HiveApache KafkaApache SparkKubernetesMicrosoft SQL ServerMLflowMongoDB.NETPandasPythonscikit-learnSQL
Categories
Data Engineering
About Schwarz Group
Schwarz Group is a German retail group that owns and operates the Lidl and Kaufland grocery chains, plus production and waste-management businesses (Schwarz Produktion and PreZero). Through its digital arm Schwarz Digits it builds and runs IT, cloud, data, and security services for its brands and external clients, and offers retail media solutions via Schwarz Media. Headquartered in Neckarsulm, it is privately held and active across Europe and beyond.