1 month ago
Greece (city not specified)Staff+
Responsibilities
- Own the production improvement loop for Context Engine by analyzing extraction automations, retrieval tools, memory, evaluations, customer feedback, and telemetry.
- Define safe iteration practices for agents and skills, including versioning, prompt and automation rollouts, regression coverage, staged evaluation, shadow evaluation, and guardrails.
- Design telemetry for agent traces, tool calls, and knowledge retrieval and use it to support evaluations, dashboards, and feedback loops in Elasticsearch.
- Partner with data scientists on golden datasets and evaluators measuring quality, latency, and cost.
- Run product experiments involving instrumentation, unattended execution, and interpretation of results.
- Review designs and pull requests, mentor engineers in evaluation-driven development, and write technical proposals that shape the roadmap.
Requirements
- 10+ years of software engineering experience, including recent experience shipping and operating AI-driven products on real production traffic.
- Demonstrated experience with evaluation-driven product improvement for agent or LLM behavior using traces, user feedback, measured fixes, and outcome analysis.
- Direct experience building agents with state and memory and safely iterating on prompts, skills, and tool behavior in production.
- Familiarity with MCP, including exposing public MCP servers and tools.
- Experience designing AI-system telemetry and using it for engineering and product decisions.
- Experience building products with public APIs and evolving data models without breaking external contracts.
- Strong backend engineering skills in Python or TypeScript, including APIs, stateful workflows, data pipelines, and production services.
- Ability to collaborate with data scientists, engineers, and product managers and communicate technical trade-offs clearly.
- Experience with agent frameworks such as LangGraph, CrewAI, or Claude Agent SDK is a bonus.
- Agentic retrieval, knowledge representation, and practical Elasticsearch experience are bonus qualifications.
Benefits
- Base salary with no variable compensation component.
- Health coverage for employees and families in many locations.
- Flexible locations and schedules for many roles within Elastic’s distributed work model.
- Generous vacation allowance.
- Elastic matches up to $2000, or local currency equivalent, for financial donations and service.
- Up to 40 hours per year for volunteer projects.
- At least 16 weeks of parental leave.
- Distributed, async-first work environment.
Tech Stack
Categories
About Elastic
Elastic builds the Elasticsearch-based platform for enterprise search, observability (logging, metrics, APM), and security (SIEM/endpoint). It sells subscriptions and Elastic Cloud managed services with usage-based pricing, combining open source and proprietary features through Elasticsearch and Kibana. Founded in 2012 and headquartered in San Francisco, Elastic is publicly traded on the NYSE and is used by more than half of the Fortune 500.