2 months ago
Toronto, Canada or London, United KingdomStaff+
Responsibilities
- Own the evaluation strategy for North across agent workflows, tool use, enterprise knowledge work and other human-AI interactions.
- Build high-quality evaluations from user feedback, production failures, privacy-preserving usage logs, internal dogfooding, customer needs and product goals.
- Create systems that continuously convert product learning and user feedback into current evaluations rather than static benchmarks.
- Analyze evaluation results, customer-derived data and product context to identify model failures and capability gaps.
- Translate North-specific needs into actionable recommendations for central modelling teams, including model selection, patches and regular model updates.
Requirements
- Experience improving LLM-powered, agent-powered or AI-product systems through evaluations, feedback loops, data curation, prompting, model adaptation or model selection.
- Strong applied machine learning judgment and the ability to reason about model behavior, evaluation validity, product outcomes and production tradeoffs.
- Deep interest in evaluation craft, including representative tasks, precise rubrics, clean data, failure analysis, regression tracking and detecting false confidence in metrics.
- Ability to work closely with users and product teams and translate qualitative signals into actionable measurement.
- Self-directed and practical approach to open-ended problems requiring technical depth and product understanding.
- Commitment to making AI systems useful in real enterprise settings rather than merely improving abstract benchmark performance.
Benefits
- Weekly lunch stipend of $75/£75 or local-currency equivalent.
- Full health and dental benefits with a separate mental-health budget.
- RRSP matching, 401K and pension scheme options.
- 100% parental-leave top-up for up to six months for either parent.
- Annual enrichment benefits covering arts and culture, fitness and wellness, quality time and workspace improvements.
- Education and learning stipend for conferences, courses and coaching.
- Six weeks of paid vacation, or 30 working days.
- Travel budget for remote employees to visit other offices and an annual company offsite.
- Remote-friendly work arrangement with offices in Toronto, London, New York City, San Francisco, Montreal, Paris, Berlin and Seoul.
- Coworking benefit for employees who are not near an office and a $500 home-office stipend.
- Office employees receive a daily lunch program, snacks and community and social events.
Categories
About Cohere
Cohere builds large language models and an enterprise AI platform that companies use for search, summarization, and workflow automation, delivered via API or private deployments. Founded in 2019 and headquartered in Toronto, it focuses on multilingual models, data controls, and options to run across major clouds or on-premises. The business is privately held and serves security- and compliance-sensitive organizations.