1 day ago
Amsterdam, NetherlandsSenior
Base Salary
$225k - $255k/yr
Responsibilities
- Build evaluation harnesses and benchmarks using tracked pricing outcomes as ground truth.
- Automate expert-review workflows and persona-training pipelines.
- Develop AI personas that simulate B2B buying committees using usage data and call transcripts.
- Own LLM routing across Anthropic and Google while balancing cost, latency, and quality.
- Maintain infrastructure and data-residency boundaries for regional model calls.
- Extend the MCP server used by LLM agents to support agent-driven product features.
- Use typed pricing ontologies and schemas to keep model outputs structured and auditable.
- Identify and remediate latency, data drift, and cold-start issues.
Requirements
- 8 or more years of engineering experience with strong, recent production LLM depth.
- End-to-end experience shipping and owning LLM-powered product features through production monitoring.
- Experience building LLM evaluations, observability, eval harnesses, and prompt baselines for release gating.
- Hands-on MCP or LLM-agent tool-building experience and familiarity with LangChain, LlamaIndex, Braintrust, or OpenRouter.
- Experience with structured data models and typed schemas such as Pydantic.
- Experience operating under data residency, SOC2, and GDPR constraints in audited domains such as pricing, billing, or payments.
- Strong communication skills for explaining non-deterministic systems to clients and partners.
- Product-engineering judgment for scoping and shipping pragmatic solutions in a lean environment.
- Must be authorized to work in the United States.
Benefits
- On-site work in Amsterdam, North Holland, Netherlands.
- Visa sponsorship is not available for this role.