3 months ago
Responsibilities
- Build and improve student models through supervised fine-tuning and knowledge distillation, including serving, generation termination, throughput, and GPU inference economics.
- Design ML evaluation methodology, including judge design and offline and online metrics.
- Develop agentic memory, buyer and user representations, profile generation, and personalization signals.
- Build whole-page optimization, page composition, ranking, blending, propensity, and predictive models for agent surfaces.
- Apply rigorous experimentation, including holdouts, A/B testing, counterfactual evaluation, and off-policy evaluation.
- Integrate tool calling, structured output, and retrieval-augmented patterns into the LLM orchestration stack.
- Own ambiguous, high-leverage ML problems end to end and act as a technical reference point for the team.
- Contribute to ML architecture discussions across Shopify and external model-lab partnerships.
Requirements
- Production experience shipping LLM systems at meaningful scale to real users.
- Hands-on experience with model training, supervised fine-tuning, and knowledge distillation.
- Experience with inference serving and throughput economics in addition to training.
- Strong evaluation methodology skills covering judge design and offline and online metrics.
- Ability to independently drive ambiguous, difficult problems to measured outcomes as a senior individual contributor.
- Breadth across distillation, memory, optimization, propensity modeling, ranking, and evaluation.
- Strong communication and collaboration skills.
- Ability to work with significant overlap with North America time zones.
- Preferred experience in e-commerce, marketplaces, commerce search, recommender systems, or advertising.
- Preferred experience with vLLM or similar engines, structured and constrained decoding, KV-cache, GPU throughput economics, and latency budgets.
- Preferred reinforcement learning experience including RLHF, RLAIF, or preference optimization.
- Preferred experience with propensity modeling, ranking, classical applied ML, calibration, position-bias correction, and counterfactual evaluation.
- Preferred experience working with frontier AI products and agentic systems.
Benefits
- Remote role based in North America.
Categories
About Shopify
Shopify is a leading global commerce company, providing trusted tools to start, grow, market, and manage a retail business of any size. Shopify makes commerce better for everyone with a platform and services that are engineered for reliability, while delivering a better shopping experience for consumers everywhere. Shopify powers millions of businesses in more than 175 countries and is trusted by brands such as Allbirds, Gymshark, PepsiCo, Staples, and many more. Find all our jobs here: www.shopify.com/careers