Tech Lead, Agent Eval Platform
ServiceNow7 days ago
Responsibilities
- Lead engineers and own end-to-end delivery of agent evaluation platform projects.
- Build orchestration services for multi-turn agent scenarios, including environment setup, simulation, validation, scoring, retries, isolation, scheduling, and teardown.
- Develop versioned datasets, specifications, reports, run comparisons, and self-service evaluation workflows.
- Lead the transition to OpenTelemetry-native observability with queryable trajectory spans, context propagation, prompt and completion capture, fault attribution, and cross-run diffing.
- Build stateful simulation environments with persistent datastores, hermetic setup and teardown, LLM-driven user simulators, deterministic state-machine simulators, and API contract-testing mocks.
- Establish reliability floors and SLOs for the evaluation harness.
- Lay the foundation for using evaluation signals to optimize and train agent behavior.
Requirements
- 10+ years building production backend or infrastructure systems.
- Strong proficiency in Python or Go, ideally both.
- Experience designing and operating systems that handle real traffic at scale.
- Experience in at least three areas including distributed systems, orchestration and workflow runtimes, observability internals, concurrent and asynchronous programming, data-intensive pipelines, and gRPC/protobuf service design.
- Ability to technical-lead other engineers and own end-to-end project delivery.
- Comfort making nondeterministic systems measurable and working through ambiguous problems without established textbook solutions.
- Experience with scheduling, retries, high-concurrency systems, trace data, data pipelines, or service and interface design is relevant to the role.
Benefits
- Regular employee position.
- Required in-office work persona in the AMS - North America and Canada region.
- ServiceNow/Moveworks distributed-work approach includes flexible, remote, or required-in-office personas depending on role and location.
- Equal opportunity employer with reasonable accommodation support.
Categories
About ServiceNow
ServiceNow (NYSE: NOW) makes the world work better for everyone. Our cloud-based platform and solutions help digitize and unify organizations so that they can find smarter, faster, better ways to make work flow. So employees and customers can be more connected, more innovative, and more agile. And we can all create the future we imagine. The world works with ServiceNow. For more information, visit www.servicenow.com.