13 days ago
Madrid, SpainSenior
Responsibilities
- Build and operate production-grade data and AI pipelines for document processing in a micro-batch architecture.
- Design and implement data extraction, retrieval, knowledge discovery, and multimodal search solutions.
- Apply AI/ML, NLP, LLM-based RAG architectures, embeddings, and vector search to enterprise knowledge platforms.
- Develop and support AI-powered search and knowledge management capabilities involving text, images, and scanned documents.
- Use Apache Spark with PySpark or Scala for performance tuning, data skew handling, and efficient cluster utilization.
- Own solutions end to end, including design, implementation, production support, and continuous improvement.
- Introduce technologies that improve service reliability and cost efficiency.
- Collaborate with subject matter experts and international team members in English.
- Evaluate and benchmark LLMs for enterprise search and knowledge management where appropriate.
Requirements
- Degree in computer science, software engineering, mathematics, or a similar field, or similar experience.
- Experience building and operating production-grade big data pipelines on Databricks.
- Experience applying AI/ML and NLP techniques to information extraction, retrieval, or knowledge discovery.
- Experience with LLM-based RAG architectures, embeddings, and vector search.
- Hands-on Apache Spark expertise, including PySpark or Scala, performance tuning, data skew handling, and efficient cluster usage.
- Solid Python programming skills.
- Experience with Elasticsearch or OpenSearch.
- Experience working in cloud-based DevOps environments, including Azure, Azure DevOps, and CI/CD pipelines.
- Fluent English communication skills.
- Experience with Azure Data Factory, Kubernetes, Azure Storage Accounts, or ADLS2 is preferred.
- Experience with multimodal document processing and LLM evaluation or benchmarking is preferred.
- Strong problem-solving skills and the ability to develop pragmatic, scalable solutions are required.
Benefits
- Flexible work schedule and home-office options.
- Learning and development opportunities.
- 23 holiday days per year, plus 5 additional readjustment days and 2 cultural days.
- Collaborative, trustful, and innovative work environment.
- International team and global project opportunities.
- Relocation assistance to Madrid.
- Secure work environment focused on health, safety, and wellbeing.
Tech Stack
Categories
AI ApplicationsData Engineering
About BASF SE
BASF SE is a public chemical manufacturer that produces basic chemicals, advanced materials, surface technologies, and agricultural solutions for customers in industries such as automotive, construction, and consumer goods, as well as farmers. It sells globally through a portfolio of business segments and R&D-driven product lines. Founded in 1865 and headquartered in Ludwigshafen, Germany, BASF is listed on the Frankfurt Stock Exchange.