13 hours ago
Remote, United States or Remote, CanadaSenior
Base Salary
$160k - $190k/yr
Responsibilities
- Design, build, and ship portable production AI agents with LangGraph.
- Own agent architecture, graph and tool design, prompt and context engineering, durable execution, retries, and cost and latency budgets.
- Build integrations with internal systems using MCP, direct APIs, and agent-to-agent interfaces.
- Deliver human-in-the-loop review flows with confidence disclosure, correction paths, and override capabilities.
- Build and tune retrieval with chunking, hybrid retrieval, grounding, and source citations.
- Create agent evaluations using golden datasets, LLM-as-judge and deterministic scorers, regression suites, and shared evaluation tooling.
- Diagnose and correct retrieval, prompt, tool, model, and ground-truth quality failures.
- Implement agent safety behaviors including prompt-injection resistance, PII handling, refusal, and escalation paths.
- Partner with Product, Platform Engineering, Data Platform, and domain teams on shipped behavior, deployment, curated datasets, analytics, and long-term operations.
- Transition reference agents to domain teams and contribute to shared agent quality standards.
Requirements
- 7+ years of professional software engineering experience, including 2+ years building LLM-powered systems that reached production and real users.
- Strong Python and service-stack expertise, including FastAPI, Pydantic, or equivalent technologies, with experience in testing, code review, CI/CD, and production ownership.
- Production experience with an agent orchestration framework; LangGraph is strongly preferred, while LangChain, OpenAI or Claude Agents SDKs, or equivalent frameworks are considered.
- Hands-on depth with at least one frontier model API.
- Hands-on experience with LLM evaluation, including golden datasets, LLM-as-judge, deterministic scorers, regression testing, and release gating.
- Experience with MCP tool servers or comparable tool and function-calling protocols and multi-agent patterns.
- Production RAG and retrieval experience, including chunking, hybrid retrieval, grounding, citation, and retrieval-failure diagnosis.
- Experience with LLM observability and tracing using LangSmith, Langfuse, Arize, or equivalent tools, plus prompt and version management.
- Sound judgment regarding hallucination, prompt injection, silent degradation, and acceptable versus unacceptable failures.
- Preferred experience with graph-backed agent memory or knowledge graphs such as Neo4j; vector and hybrid retrieval stores such as pgvector, Pinecone, Weaviate, or Qdrant; Celery/Redis or equivalent asynchronous task orchestration; and document intelligence, OCR, layout-aware parsing, and structured extraction from long documents.
Benefits
- Competitive compensation and comprehensive benefits including health, dental, vision, a 401(k) plan with company match, and generous paid time off.
- Flexible remote, hybrid, or in-office work arrangements.
- Inclusive and collaborative workplace culture.
- Equal opportunity employment regardless of the protected characteristics listed in the posting.
Tech Stack
Categories
