13 hours ago
Base Salary
$160k - $250k/yr
Responsibilities
- Own the core agent intelligence layer for multi-step workflows across desktop CAD, simulation, and PLM tools.
- Define evaluation frameworks, baselines, task-completion metrics, token budgets, and cost-per-workflow tracking.
- Build reproducible evaluation infrastructure grounded in validated real user stories and translate stories into benchmark test cases.
- Make architecture decisions covering tool calling, state management, error recovery, model routing, and context management.
- Lead and mentor a small team while writing production code, reviewing architecture, and improving engineering standards.
- Collaborate with integrations, product, domain experts, and customers during proofs of concept.
Requirements
- 7+ years of software engineering experience, including at least 2 years building LLM-based agents that take real-world actions.
- Exceptional depth in agentic systems, including model selection, context and window management, retrieval, tool calling, and orchestration.
- Experience building evaluation and benchmarking frameworks for task completion, cost efficiency, and failure modes.
- Strong Python proficiency and hands-on familiarity with LLM tooling, function calling, tool APIs, observability and tracing, and evaluation frameworks.
- Experience shipping AI or LLM tooling over proprietary engineering data or desktop engineering software such as CAD, PLM, or simulation systems; general-purpose chatbot or web-app RAG experience alone is insufficient.
- Experience setting technical direction and reviewing code for small engineering teams while continuing to write production code.
- Experience with desktop automation or programmatic application control such as COM interfaces.
- Background in mechanical engineering, CAD/CAE, PLM, or an adjacent engineering-software domain.
- Familiarity with deploying agents on locked-down enterprise workstations and managing related security and operational constraints.
- Published work, public benchmarks, or open-source contributions in agentic AI are preferred.
Benefits
- Equity is included in the compensation package.
- On-site role in San Francisco, California.