2 months ago
Remote, United StatesStaff+
Responsibilities
- Own and grow the agentic function across internal data-engineering and data-science agents and external customer- and partner-facing agents.
- Design, build, and deploy production LLM agents using MCP, function calling, tiered tool access, approval gates, human-in-the-loop controls, and rollback mechanisms.
- Build evaluation harnesses with offline and online evaluation, regression testing, guardrails, and observability and tracing.
- Own context and retrieval engineering, including context assembly, grounding, chunking strategies, structured retrieval, and hallucination reduction.
- Implement graph-based knowledge and retrieval systems using GraphRAG, property graphs, ontology and semantic layers, and related approaches.
- Partner with applied science and data/ML teams on pipelines, embeddings, and vector stores without owning model training.
- Communicate agent architecture, trade-offs, and roadmap to executives and investors while writing production code and hiring and developing the team.
Requirements
- At least 5 years of experience shipping production software, with strong full-stack or backend engineering skills and experience with production APIs, data models, testing, and CI.
- Demonstrated experience building and shipping multi-step, tool-calling, stateful LLM agents with orchestration such as LangGraph to real users.
- Ability to explain agent control loops beyond the framework used.
- Experience building production evaluation harnesses for agentic systems using tools such as MLflow, LangSmith, or custom systems, with knowledge of determinism, drift, and guardrails.
- Experience owning production retrieval systems and evaluating retrieval quality, including chunking versus structured retrieval trade-offs.
- Experience shipping agents that take consequential real-world actions and designing approval, guardrail, and rollback architectures.
- Track record leading an agentic initiative or team end-to-end from architecture through production and communicating agent systems to executives and investors.
- Preferred experience with graph-backed retrieval, knowledge graphs, ontologies, property graphs, and triple stores.
- Preferred experience fine-tuning or distilling open-source models with LoRA or QLoRA, or strong context and prompt optimization experience.
- Preferred experience serving or operating Llama, Qwen, or Mistral models through Databricks, vLLM, or similar systems and evaluating self-hosted versus hosted API trade-offs.
- Python is the core language; TypeScript, Node.js, or Go experience is a plus.
Benefits
- Remote work arrangement.
- Opportunity to help launch and scale mobility-focused software and hardware ventures within a venture studio.
Tech Stack
Categories
