Senior Staff Software Engineer - AI/ML Inference Engineering
ServiceNow1 day ago
Base Salary
$181k - $317k/yr
Responsibilities
- Lead the architecture, optimization, and operational excellence of the AI/ML inference platform.
- Own the end-to-end inference lifecycle, including model onboarding and deployment of optimized inference engines across multiple production backends.
- Build and operate inference services on Kubernetes infrastructure using model-serving and runtime technologies.
- Architect an intelligent Envoy and ext_proc gateway layer for access control, observability, and advanced request routing.
- Deploy, scale, and operate production services with Helm-based deployments and repeatable, rollback-safe release pipelines.
- Own metrics, logs, tracing, alerts, service-level objectives, production debugging, incident resolution, on-call participation, and root-cause analysis.
Requirements
- 8+ years of backend software engineering experience.
- 4+ years of experience designing and operating distributed systems.
- Bachelor’s degree in computer science, software engineering, or a closely related technical field.
- Expertise in Go or Python, object-oriented programming, design patterns, efficient algorithms, and modular software development.
- Experience with model serving and runtimes such as vLLM, TensorRT-LLM, or Triton Inference Server.
- Knowledge of core AI/ML techniques and algorithms, unit testing, profiling, and code tuning.
- Experience building products using challenging algorithms and producing efficient, high-quality software.
- Experience with production Kubernetes environments, Helm deployments, observability, incident response, and service-level objectives is desired.
- A master’s degree may offset up to one year of the experience minimum, and equivalent practical experience may be considered when technical depth is demonstrable.
- Experience with AI coding assistants such as GitHub Copilot or Cursor, AI-assisted debugging, trace analysis, and data privacy considerations is expected.
Benefits
- Base pay is $181,200 - $317,100, plus equity when applicable and variable or incentive compensation.
- Benefits include health plans, flexible spending accounts, a 401(k) plan with company match, ESPP, matching donations, flexible time away, and family leave programs.
- The role uses a flexible work persona within ServiceNow’s distributed work approach.
- Employee type is regular, and the listed region is North America and Canada.
Tech Stack
Categories
About ServiceNow
ServiceNow builds a cloud platform for enterprise digital workflows, covering IT service management, customer service, HR service delivery, security operations, and operations management, plus tools for custom app development. It sells subscription SaaS to large organizations and public-sector agencies to automate processes and connect data across systems. Founded in 2004 and headquartered in Santa Clara, California, ServiceNow is a public company listed on the NYSE under the ticker NOW.