14 days ago
Base Salary
$240k - $415k/yr
Responsibilities
- Embed with customer engineering teams to architect integrations between Cohere AI models and complex production environments.
- Design model-serving topologies covering multi-tenant architectures, intelligent routing, multi-cluster environments, and cross-datacenter failover.
- Optimize private and AI-in-a-box deployments while balancing Kubernetes scale and model performance.
- Benchmark and profile customer LLM workloads and diagnose performance degradation using metrics, dashboards, and distributed logs.
- Validate and support Cohere models on GPUs, TPUs, AMD accelerators, and custom ASICs.
- Translate customer deployment challenges into platform upgrades and contribute feedback to core product features.
- Help customers establish production reliability practices, including deployment patterns, SLOs, and observability frameworks.
- Provide technical leadership for ambiguous customer problems and write production code while managing deployment blockers.
Requirements
- 8+ years of engineering experience running production infrastructure, designing large-scale distributed systems, or deploying enterprise software in complex environments.
- Deep production Kubernetes experience across multi-cluster and hybrid or multi-cloud environments, including GCP, Azure, AWS, and OCI.
- Strong understanding of LLM architecture and inference dynamics, including attention mechanisms, prefill, and decode phases.
- Hands-on experience with modern inference engines and optimization stacks such as vLLM, TensorRT-LLM, lmd, or Dynamo.
- Familiarity with distributed serving techniques including prefill/decode disaggregation, distributed KV caching, and multi-GPU workloads.
- Proven customer-facing technical leadership and experience scoping ambiguous problems and guiding executive technical stakeholders.
- Ability to write production code while handling customer-side deployment politics or blockers.
- Strong programming foundations in Python, Go, or C++, with comfort working across backend data pipelines and infrastructure tuning.
Benefits
- Weekly lunch stipend of $75/£75 or local-currency equivalent.
- Full health and dental benefits with a separate mental-health budget.
- RRSP matching, 401(k), and pension scheme benefits.
- 100% parental-leave top-up for up to six months for either parent.
- Annual arts and culture, fitness and wellness, quality-time, workspace-improvement, education, learning, conference, course, and coaching benefits.
- Six weeks of paid vacation, totaling 30 working days.
- Remote-friendly work arrangement with offices in Toronto, London, New York City, San Francisco, Montreal, Paris, Berlin, and Seoul.
- Remote employees receive travel support for other offices, an annual company offsite, a co-working benefit, and a $500 home-office stipend; office employees receive lunch, snacks, and community events.
Categories
Forward Deployed
About Cohere
Cohere builds large language models and an enterprise AI platform that companies use for search, summarization, and workflow automation, delivered via API or private deployments. Founded in 2019 and headquartered in Toronto, it focuses on multilingual models, data controls, and options to run across major clouds or on-premises. The business is privately held and serves security- and compliance-sensitive organizations.