19 hours ago
Base Salary
$165k - $226k/yr
Responsibilities
- Build agent harnesses supporting planning, tool use, memory, context compaction, interruption, recovery, approvals, and long-running execution.
- Integrate OpenAI and Anthropic model APIs and select models based on measured quality, latency, cost, and failure handling.
- Connect agent harnesses to AWS workflow execution and approval state while keeping authorization and sensitive writes behind application policy checks.
- Build evaluation datasets and replay tests for findings, access decisions, application setup, and AI agent onboarding.
- Measure false positives, missed risks, unsafe actions, completion rates, and costs, and convert production failures into regression tests.
- Test prompt injection, stale evidence, conflicting instructions, unsafe tool use, and tenant-boundary protection in retrieval and memory.
- Improve prompts, retrieval, tools, and model routing, and evaluate fine-tuning or other post-training methods when justified and approved.
Requirements
- Strong Python engineering skills and experience shipping an LLM application that uses tools.
- Hands-on experience building agent workflows or evaluation systems for tool-using LLM applications.
- Experience with model APIs, structured outputs, and asynchronous services, particularly OpenAI or Anthropic.
- Depth in agent execution or evaluation systems, including the ability to explain working code and failure analysis.
- Ability to distinguish plausible model responses from supported findings or verified actions.
- Experience with tools such as Claude Agent SDK, OpenAI Codex SDK, Strands Agents, LangChain Deep Agents, LangGraph, or a custom implementation is relevant.
- Extra credit includes open-source contributions, agent observability, security evaluations, synthetic evaluation data, fine-tuning, preference data, post-training evaluation, model distillation, identity and security workflows, or foundation-model research experience.
Benefits
- Annual base salary range of $165,000—$226,000 USD for candidates located in the San Francisco Bay Area.
- Equity where applicable, bonus, health, dental and vision insurance, 401(k), flexible spending account, and paid leave including PTO and parental leave.
- Immersive, in-person onboarding experience and connection-focused community across Okta’s global offices.
Tech Stack
Categories
About Okta
Okta builds cloud-based identity and access management for enterprises and developers, including single sign-on, multi-factor authentication, and lifecycle management. It sells subscription SaaS as the Okta Workforce Identity and Customer Identity Clouds; the latter incorporates Auth0, acquired in 2021. Founded in 2009 and headquartered in San Francisco, Okta is a public company traded on Nasdaq.