5 hours ago
Responsibilities
- Own the evaluation strategy for North across agent workflows, tool use, enterprise knowledge work and other human-AI interactions.
- Build high-quality evaluations from user feedback, production failures, privacy-preserving usage logs, internal dogfooding, customer needs and product goals.
- Create systems that continuously convert product learning and user feedback into current evaluations rather than static benchmarks.
- Analyze evaluation results, customer-derived data and product context to identify model failures and capability gaps.
- Translate North-specific needs into actionable recommendations for central modelling teams, including model selection, patches and regular model updates.
Requirements
- Experience improving LLM-powered, agent-powered or AI-product systems through evaluations, feedback loops, data curation, prompting, model adaptation or model selection.
- Strong applied machine learning judgment and the ability to reason about model behavior, evaluation validity, product outcomes and production tradeoffs.
- Deep interest in evaluation craft, including representative tasks, precise rubrics, clean data, failure analysis, regression tracking and detecting false confidence in metrics.
- Ability to work closely with users and product teams and translate qualitative signals into actionable measurement.
- Self-directed and practical approach to open-ended problems requiring technical depth and product understanding.
- Commitment to making AI systems useful in real enterprise settings rather than merely improving abstract benchmark performance.
Benefits
- Weekly lunch stipend of $75/£75 or local-currency equivalent.
- Full health and dental benefits with a separate mental-health budget.
- RRSP matching, 401K and pension scheme options.
- 100% parental-leave top-up for up to six months for either parent.
- Annual enrichment benefits covering arts and culture, fitness and wellness, quality time and workspace improvements.
- Education and learning stipend for conferences, courses and coaching.
- Six weeks of paid vacation, or 30 working days.
- Travel budget for remote employees to visit other offices and an annual company offsite.
- Remote-friendly work arrangement with offices in Toronto, London, New York City, San Francisco, Montreal, Paris, Berlin and Seoul.
- Coworking benefit for employees who are not near an office and a $500 home-office stipend.
- Office employees receive a daily lunch program, snacks and community and social events.
Categories
About Cohere
Cohere is the leading security-first enterprise AI company. We build cutting-edge foundation models and end-to-end AI products designed to solve real-world business problems. We partner closely with companies to deliver seamless integration, full customization, and easy-to-use solutions for their workforce and customers. Our all-in-one platform offers enterprises the highest levels of data security, privacy and optionality to deploy across all major cloud providers, private cloud environments, or on-premises. HQ: 171 John Street, 2nd Floor, Toronto, ON M5T 1X3