11 hours ago
Toronto, CanadaStaff+
Responsibilities
- Implement intent-based enforcement to verify that agent runtime requests match their intended purpose.
- Apply LLM reasoning and prompt parsing to interpret prompts, tool payloads, and agent intent in real time.
- Integrate low-latency inference and semantic evaluation engines into API gateway request paths.
- Use embeddings, vector search, and zero-shot classification to score alignment between agent intent and actions.
- Design confidence-scored decision engines that integrate semantic verification with policy frameworks such as Cedar.
- Develop evaluation benchmarks, prompt-injection defenses, guardrails, schema validation, and structured-output enforcement.
- Architect scalable ML and generative AI systems with retrieval, inference, evaluation, and RAG pipelines.
- Optimize prompting, context retrieval, and RAG workflows for Claude-based systems.
- Build production evaluation pipelines measuring model quality, correctness, groundedness, and safety.
Requirements
- 8+ years of software development experience and strong programming expertise in Python.
- Familiarity with Go or TypeScript is a plus.
- Hands-on applied machine learning experience spanning feature engineering, model training, and fine-tuning.
- Experience with generative AI platforms such as AWS Bedrock, OpenAI, and Anthropic.
- Deep understanding of retrieval-augmented generation, embeddings, and knowledge-base workflows.
- Experience with LiteLLM, LangGraph, LangChain, LlamaIndex, MCP, or related AI agent frameworks.
- Familiarity with FastAPI, PyTorch, TensorFlow, Spark ML, and workflow orchestration tools such as Airflow.
- Experience defining evaluation metrics, pipelines, and feedback loops for ML and generative AI systems.
- Experience building tools or infrastructure for AI/ML applications and understanding AI-native developer workflows.
- Bachelor’s or master’s degree in computer science or a related field.
- Preferred experience includes integrating AI systems with identity, authentication, or security products; ethical AI, model risk, or compliance frameworks; and evaluation datasets, synthetic data generation, or LLM-as-a-judge methods.
Benefits
- Health, dental, and vision insurance.
- RRSP with a match, healthcare spending, telemedicine, and paid leave including PTO and parental leave.
- Hybrid work arrangement, as indicated by the posting.
- Immersive in-person onboarding experience and opportunities for talent development, connection, and community.
- The posting applies to candidates located in Canada and describes benefits according to applicable plans and policies.
Tech Stack
Categories
About Okta
Okta builds cloud-based identity and access management for enterprises and developers, including single sign-on, multi-factor authentication, and lifecycle management. It sells subscription SaaS as the Okta Workforce Identity and Customer Identity Clouds; the latter incorporates Auth0, acquired in 2021. Founded in 2009 and headquartered in San Francisco, Okta is a public company traded on Nasdaq.