2 months ago
Toronto, CanadaMid Level / Senior
Responsibilities
- Deploy and monitor classical NLP, embedding, and LLM-powered ML systems processing millions of records per day.
- Own the evaluation stack, including golden datasets, model-as-a-judge frameworks, inter-annotator agreement, and release-gating regression tests.
- Build and maintain the vector embeddings ecosystem and its retrieval, classification, and similarity patterns.
- Partner with Data Science on annotation workflows, PII scrubbing, and ground-truth pipelines.
- Improve MLOps foundations including versioning, observability, and drift detection.
- Translate ambiguous product problems into measurable AI features with clear success criteria.
Requirements
- 4–7 years of professional software or machine learning engineering experience, including 2+ years shipping ML systems to production.
- Strong Python skills and comfort with the modern data and machine learning stack.
- Hands-on experience deploying and monitoring models in AWS or GCP.
- Production experience with NLP or ML systems involving classification, NER, embeddings, ranking, similarity, or LLM-powered features.
- Practical experience evaluating ML or LLM systems using methods such as golden datasets, model-as-a-judge, inter-annotator agreement, or precision/recall.
- Ability to collaborate with data scientists and engineers and explain ideas, requirements, and tradeoffs to non-technical stakeholders.
- Experience with vector databases or retrieval systems at scale is a bonus.
- Experience with managed ML services such as SageMaker or Vertex AI is a bonus.
- Annotation workflow experience with Label Studio, Scale AI, or similar tools is a bonus.
- Familiarity with PII scrubbing patterns and privacy-by-design data handling is a bonus.
- Open-source contributions, blog posts, or talks about LLM or embedding production work are a bonus.
Benefits
- Flexible work model combining in-person and remote work.
- Learning platform, training, and tools from the first day.
- Surprise meal stipends.
- Vacation, floating and set holidays, wellness allowance, and paid parental leave.
- Medical, dental, vision, life, and disability insurance.
- Work-from-home stipend.
Tech Stack
Categories
About CreatorIQ
CreatorIQ is the operating system for creator-led growth. Trusted by more than 1,300 global brands and agencies—including Burson, Delta Air Lines, Google, LVMH, Nestlé, and Sephora—CreatorIQ unifies creator marketing across paid, owned, earned, commerce, and community into one seamless, enterprise-grade ecosystem. Powered by the Creator Graph™, CreatorIQ’s industry-leading intelligence infrastructure, along with rigorous compliance and security standards, and integrations with Meta, Snapchat, TikTok, YouTube, and more, CreatorIQ empowers brands and agencies to harness the creator economy as a strategic growth engine.