2 months ago
Base Salary
$150k - $190k/yr
Responsibilities
- Monitor production health across agent graphs and identify errors, slow runs, expensive runs, and silent failures.
- Triage issues from Slack, tickets, and go-to-market representatives, resolving smaller issues and routing others to the appropriate owner.
- Run weekly evaluation suites, investigate failures, and convert production bugs into permanent regression tests.
- Track cost and latency by model, graph, use case, and role, and recommend changes to model selection, reasoning effort, and caching.
- Track usage and adoption by representative and feature, and own the team’s weekly health report.
- Build business metrics covering reply rates, meetings booked, hours reclaimed, and return on investment.
- Build monitoring and alerting on LangSmith and document how LangChain operates its own agent for customers.
Requirements
- Strong production Python and SQL skills, with comfort working in traces, logs, and warehouse tables.
- Hands-on experience running LLM applications, including tracing, evaluations, prompt mechanics, and cache mechanics.
- SRE or production operations instincts, including understanding percentiles and service-level objectives and distinguishing noise from meaningful patterns.
- Ability to scrutinize metrics and verify what measurements actually count before publishing them.
- Clear writing skills and interest in publishing technical learnings.
- High agency and initiative to identify gaps and build solutions.
Benefits
- Medical, dental, and vision coverage.
- Flexible vacation.
- 401(k) plan.
- Meals on in-office days in the US.
- Compensation includes base salary, variable compensation for relevant roles, equity, benefits, and perks; offerings vary by role, level, and location.
- Team members in the EU, UK, and APAC receive locally competitive benefits aligned with regional norms and regulations.
Categories
About LangChain
LangChain builds open-source frameworks and a commercial platform for developing, evaluating, deploying, and operating LLM-powered agents and applications. Its LangSmith suite provides observability, testing, deployment, and governance for enterprise AI workloads; customers include Nvidia, LinkedIn, Workday, and Bridgewater. Privately held and headquartered in San Francisco, the company was founded in 2022.