5 days ago
Base Salary
$160k - $250k/yr
Responsibilities
- Lead development of the agent intelligence layer for multi-step workflows across CAD, simulation, and PLM software.
- Own the product loop from user story definition through implementation and benchmarking against real engineering workflows.
- Define evaluation frameworks, baselines, task success metrics, failure analysis, and cost-efficiency measures.
- Design evaluation infrastructure and translate validated user stories into reproducible agent benchmarks.
- Own agent architecture decisions involving tool calling, state management, error recovery, model routing, and context management.
- Lead user story mapping and validation through interviews and collaboration with domain experts.
- Act as a player-coach by writing production code, reviewing designs, unblocking the team, and raising engineering standards.
- Collaborate with integrations, product, and customers during proofs of concept to align agent behavior with real-world usage.
- Set technical direction for a small team of AI engineers, a user researcher, and domain expert contractors.
Requirements
- 7+ years of software engineering experience, including at least 2 years building and shipping real-world agentic LLM systems.
- Deep experience with LLM application architecture, including model selection, context-window management, retrieval strategies, tool-calling frameworks, and orchestration patterns.
- Strong evaluation and benchmarking experience involving task completion rates, cost efficiency, and failure-mode analysis; familiarity with SWE-bench, GAIA, or τ-bench is a plus.
- A record of shipping AI systems with measurable outcomes rather than only demos or prototypes.
- Strong Python skills and hands-on familiarity with function-calling, tool-use APIs, tracing and observability tools, and evaluation frameworks.
- Technical leadership experience directing small teams of 3–6 engineers and making architecture decisions and meaningful code reviews.
- Hands-on experience with mechanical engineering software such as CAD, CAE, PLM, or simulation tooling, either as a builder or power user.
- Experience shipping AI or LLM tooling over proprietary engineering data or desktop engineering software, such as agents or MCP servers over CAD/PLM APIs or retrieval systems over engineering repositories or schematics.
- Familiarity with enterprise deployment constraints on locked-down corporate workstations.
- Desktop automation or programmatic application control experience, such as COM, is a strong plus.
- Published work, open-source contributions, or benchmark contributions in agentic AI are a plus.
Benefits
- On-site role in San Francisco, California, USA.
- Visa sponsorship is not available for this role.