3 months ago
Paris, FranceSenior / Staff+
Responsibilities
- Define and own the evaluation strategy for the AI agentic system, including metrics, protocols, datasets, and tooling.
- Implement and maintain automated evaluation pipelines monitoring model quality, safety, and alignment across iterations.
- Run systematic experiments assessing reasoning, factuality, robustness, and user experience.
- Collaborate with model developers and research scientists to provide insights and drive iterative improvement.
- Contribute to research and internal knowledge sharing on LLM evaluation methodologies and best practices.
- Work cross-functionally with Machine Learning Engineers, Product Engineers, and Medical Experts.
Requirements
- MSc or PhD in Computer Science, Machine Learning, Data Science, or a related field.
- At least 7 years of hands-on experience working with large language models such as GPT, Claude, Llama, or BERT-like architectures.
- Proven experience evaluating agentic or reasoning systems, including autonomous agents, tool-using LLMs, dialogue systems, or task-oriented assistants.
- Strong experience in experiment design, metric definition, and evaluation automation.
- Ability to bridge research and production and influence modeling and product decisions.
- Excellent communication skills and a collaborative mindset.
- Experience in the clinical or medical domain and sensitivity to ethical or regulatory challenges in healthcare AI are preferred.
Benefits
- Free health insurance for the employee and children.
- One additional month of parental leave through the Parent Care Program.
- Free mental health and coaching services through Moka.care.
- Support package for caregivers and workers with disabilities, including remote-policy adaptation, extra medical leave days, and psychological support.
- Ability to work from EU countries and the UK for up to 10 days per year.
- Work Council subsidy for part of sport club membership or creative class costs.
- Up to 14 days of RTT.
- Lunch voucher with a Swile card.
- Permanent, full-time position with a hybrid workplace in the Levallois office.
- Start date is as soon as possible.
Tech Stack
Categories
About Doctolib
Doctolib builds a digital healthcare platform for patients and medical professionals, combining online appointment booking, teleconsultations, and practice-management software. It monetizes via SaaS subscriptions to healthcare providers, while access for patients is free. Founded in 2013 and headquartered in Levallois‑Perret, the privately held company operates across France, Germany, Italy, and the Netherlands and is certified as a B Corp.