13 hours ago
Remote, Canada or Toronto, CanadaSenior
Base Salary
$72k - $139k/yr
Responsibilities
- Optimize decode latency, throughput, and cost for LLM inference on AWS Trainium platforms while validating correctness.
- Implement and improve speculative decoding and native multi-token prediction for Qwen-family models in vLLM on AWS Neuron.
- Design reproducible performance tests against GPU platforms and produce latency-throughput and price-performance comparisons.
- Benchmark and migrate workloads across Trainium generations and scale long-context serving across multi-chip systems for contexts of 100K tokens or more.
- Onboard additional hybrid state-space and mixture-of-experts model architectures.
- Profile and optimize accelerator workloads at the framework, compiler, or kernel level.
- Create runbooks, implementation guides, and benchmark documentation, and support technical workshops for customers.
Requirements
- 5+ years of experience in machine learning or systems engineering, including 2+ years focused on production model inference or serving.
- Strong proficiency in Python and PyTorch, including the ability to modify implementations involving attention, mixture-of-experts routing, and KV-cache management.
- Hands-on experience with an LLM serving framework such as vLLM, TensorRT-LLM, SGLang, or Triton Inference Server.
- Deep understanding of LLM inference concepts including prefill, decode, continuous batching, tensor parallelism, and expert parallelism.
- Experience profiling and optimizing accelerator workloads and designing rigorous performance experiments.
- Strong written and verbal communication skills for explaining technical trade-offs and business impact.
- Preferred experience with AWS Trainium or Inferentia, AWS Neuron SDK, accelerator migrations, speculative decoding, and kernel development using NKI, Triton, or CUDA.
Benefits
- Remote anywhere in Canada.
- Regular, full-time employment with a comprehensive benefits package.
- Paid time off, including vacation, company holidays, personal days, and sick leave, subject to employee grade and policy.
- Medical, dental, and vision coverage, or provincial healthcare coordination in Canada.
- Retirement savings plans, including RRSP in Canada.
- Life and disability insurance and employee assistance programs.
- Additional benefits may be provided according to local policy and eligibility.
Categories
About Capgemini
Capgemini is a global IT services and consulting firm that delivers strategy, cloud, AI, software engineering, and managed services to large enterprises and public-sector clients. Founded in 1967 and headquartered in Paris, it is publicly traded on Euronext Paris and operates in 50+ countries. The group expanded its engineering capabilities by acquiring Altran in 2020, now operating as Capgemini Engineering.