Staff Machine Learning Engineer
ServiceNow7 hours ago
Base Salary
$176k - $308k/yr
Responsibilities
- Design and build the agent execution harness for orchestration, context management, tool invocation, retries, and execution-state handling.
- Own runtime fault tolerance, latency, throughput, and reliability for enterprise-scale workflows.
- Implement tracing, cost attribution, latency visibility, and production observability for agent behavior.
- Build prompt management systems with versioning, templating, and systematic evaluation.
- Design and maintain unit, integration, and production evaluation frameworks to measure agent quality and prevent regressions.
- Integrate and abstract over frontier LLMs, including model routing, fallback strategies, cost management, and latency optimization.
- Establish technical standards for agentic design, production AI practices, system boundaries, architecture, and code quality.
- Provide technical leadership through architecture decisions, code reviews, and coaching.
Requirements
- 7+ years of experience building production software systems with demonstrated reliability, performance, and scalability expertise.
- Hands-on experience shipping generative AI products used by production customers, beyond prototypes or basic LLM API integrations.
- Strong understanding of LLM behavior, failure modes, context constraints, and prompt design at scale.
- Experience with Python, Kubeflow, SageMaker, Vertex AI, AzureML, and SQL.
- Practical experience designing, versioning, and evaluating prompts across model updates or A/B evaluation cycles.
- A demonstrated track record designing and shipping evaluation suites used for production quality decisions.
- Experience reasoning about model routing, inference cost, latency, and system-level efficiency tradeoffs.
- Strong software engineering fundamentals in distributed systems, API design, and testing.
- Ability to work effectively in fast-moving, ambiguous, startup-like AI product environments.
- Preferred experience with multi-agent coordination patterns such as A2A and MCP.
- Preferred familiarity with LangChain, LlamaIndex, or similar agent frameworks.
- Preferred experience shipping enterprise AI systems and using AI observability, logging, monitoring, reliability engineering, and production troubleshooting tools.
Benefits
- Base pay of $176,100 - $308,200, plus equity when applicable, variable or incentive compensation, and benefits.
- Health plans including flexible spending accounts, a 401(k) plan with company match, ESPP, matching donations, flexible time away, and family leave programs.
- Flexible, remote, or required-in-office work personas may apply depending on the role and location.
- Regular employee position in the AMS - North America and Canada region with a flexible work persona.
Categories
About ServiceNow
ServiceNow builds a cloud platform for enterprise digital workflows, covering IT service management, customer service, HR service delivery, security operations, and operations management, plus tools for custom app development. It sells subscription SaaS to large organizations and public-sector agencies to automate processes and connect data across systems. Founded in 2004 and headquartered in Santa Clara, California, ServiceNow is a public company listed on the NYSE under the ticker NOW.