2 months ago
Paris, FranceSenior / Staff+
Responsibilities
- Define and own the evaluation strategy for the AI agentic system, including metrics, protocols, datasets, and tooling.
- Implement and maintain automated evaluation pipelines monitoring model quality, safety, and alignment across iterations.
- Run systematic experiments assessing reasoning, factuality, robustness, and user experience.
- Collaborate with model developers and research scientists to provide insights and drive iterative improvement.
- Contribute to research and internal knowledge sharing on LLM evaluation methodologies and best practices.
- Work cross-functionally with Machine Learning Engineers, Product Engineers, and Medical Experts.
Requirements
- MSc or PhD in Computer Science, Machine Learning, Data Science, or a related field.
- At least 7 years of hands-on experience working with large language models such as GPT, Claude, Llama, or BERT-like architectures.
- Proven experience evaluating agentic or reasoning systems, including autonomous agents, tool-using LLMs, dialogue systems, or task-oriented assistants.
- Strong experience in experiment design, metric definition, and evaluation automation.
- Ability to bridge research and production and influence modeling and product decisions.
- Excellent communication skills and a collaborative mindset.
- Experience in the clinical or medical domain and sensitivity to ethical or regulatory challenges in healthcare AI are preferred.
Benefits
- Free health insurance for the employee and children.
- One additional month of parental leave through the Parent Care Program.
- Free mental health and coaching services through Moka.care.
- Support package for caregivers and workers with disabilities, including remote-policy adaptation, extra medical leave days, and psychological support.
- Ability to work from EU countries and the UK for up to 10 days per year.
- Work Council subsidy for part of sport club membership or creative class costs.
- Up to 14 days of RTT.
- Lunch voucher with a Swile card.
- Permanent, full-time position with a hybrid workplace in the Levallois office.
- Start date is as soon as possible.
Tech Stack
Categories
About Doctolib
We’re a B Corp company committed to build together the healthcare we all dream of. More than 2,900 Doctolibers in France, Germany, Italy, and the Netherlands are helping to improve the daily lives of care teams with a new generation of technologies and services. At the same time, Doctolib is helping over 80 million patients to effectively manage their health and provide quick and easy access to optimal healthcare.