Staff Machine Learning Engineer
ServiceNow7 hours ago
Base Salary
$176k - $308k/yr
Responsibilities
- Design and build the agent execution harness for routing inputs, managing context, invoking tools, handling retries, and exposing execution state across multi-step workflows.
- Own runtime fault tolerance, latency, throughput, and production reliability for enterprise workflows.
- Instrument agent systems with tracing, cost attribution, and latency visibility.
- Build prompt management systems for prompt versioning, templating, and systematic evaluation.
- Design and own unit, integration, and production evaluation frameworks that detect regressions and guide quality decisions.
- Integrate and abstract over frontier LLMs, including model routing, fallback strategies, cost management, and latency optimization.
- Set technical standards through architecture decisions, code reviews, coaching, and agentic design guidance.
- Define system boundaries among tool calls, sub-agents, hardcoded paths, and human escalation.
Requirements
- 7+ years of experience building production software systems with a track record in reliability, performance, and scalability.
- Hands-on experience shipping production generative AI products and owning AI-powered features used by customers.
- Strong understanding of large language model failure modes, context constraints, and prompt design.
- Experience with Python, Kubeflow, SageMaker, Vertex AI or AzureML, and SQL.
- Practical experience designing, versioning, and evaluating prompts across model updates or A/B evaluation cycles.
- Demonstrated experience designing and shipping evaluation suites used to make production AI quality decisions.
- Experience reasoning about model routing, inference cost, and latency tradeoffs.
- Strong software engineering fundamentals in distributed systems, API design, and testing.
- Ability to work in fast-moving, ambiguous, startup-like AI product environments.
- Preferred experience with multi-agent coordination patterns such as A2A and MCP.
- Preferred familiarity with LangChain, LlamaIndex, or similar agent frameworks.
- Preferred experience shipping enterprise AI systems and using AI observability, logging, monitoring, reliability engineering, and production troubleshooting tools.
Benefits
- Base pay of $176,100 - $308,200, plus equity where applicable and variable or incentive compensation.
- Health plans, flexible spending accounts, a 401(k) plan with company match, ESPP, matching donations, flexible time away, and family leave programs.
- Flexible distributed-work persona, with eligibility for flexible, remote, or required-in-office arrangements determined by role and location.
- Regular employee position in North America and Canada.
Categories
About ServiceNow
ServiceNow builds a cloud platform for enterprise digital workflows, covering IT service management, customer service, HR service delivery, security operations, and operations management, plus tools for custom app development. It sells subscription SaaS to large organizations and public-sector agencies to automate processes and connect data across systems. Founded in 2004 and headquartered in Santa Clara, California, ServiceNow is a public company listed on the NYSE under the ticker NOW.