Senior Staff Software Engineer - AI/ML Inference Engineering
ServiceNow17 hours ago
Base Salary
$201k - $352k/yr
Responsibilities
- Lead the architecture, optimization, and operational excellence of the AI/ML inference platform.
- Own the inference lifecycle from onboarding models to deploying optimized inference engines across multiple production backends.
- Architect an intelligent gateway using Envoy and ext_proc for access control, observability, and advanced request routing.
- Deploy, scale, and operate services in Kubernetes environments, including Helm-based deployments.
- Build meaningful metrics, logs, tracing, and customer-impact alerts for production services.
- Debug and resolve production incidents, participate in on-call, conduct root-cause analyses, and operate against defined service-level objectives.
Requirements
- 8+ years of backend software engineering experience.
- 4+ years designing and operating distributed systems.
- Expertise in Go or Python, object-oriented programming, design patterns, and efficient algorithms.
- Experience with model serving and runtimes such as vLLM, TensorRT-LLM, or Triton Inference Server.
- Experience building products involving challenging algorithms and efficient, modular, high-quality software.
- Knowledge of core AI/ML techniques and algorithms, unit testing, profiling, and code tuning.
- Bachelor’s degree in computer science, software engineering, or a closely related technical field; equivalent practical experience may be considered, and a master’s degree may offset up to one year of experience.
- Experience deploying and operating production Kubernetes services, Helm deployments, and repeatable release pipelines.
- Ability to independently debug production incidents and own observability, on-call participation, root-cause analysis, and service-level objectives.
- Confident use of AI coding assistants such as GitHub Copilot or Cursor, with rigorous review of generated code and understanding of prompt data privacy.
Benefits
- Base pay of $201,300 - $352,300, plus equity where applicable, variable or incentive compensation, and benefits.
- Health plans including flexible spending accounts, a 401(k) plan with company match, ESPP, matching donations, flexible time away, and family leave programs.
- Flexible work persona within ServiceNow’s distributed work model, subject to role and location eligibility.
- Regular employee position in the AMS - North America and Canada region.
Tech Stack
Categories
About ServiceNow
ServiceNow builds a cloud platform for enterprise digital workflows, covering IT service management, customer service, HR service delivery, security operations, and operations management, plus tools for custom app development. It sells subscription SaaS to large organizations and public-sector agencies to automate processes and connect data across systems. Founded in 2004 and headquartered in Santa Clara, California, ServiceNow is a public company listed on the NYSE under the ticker NOW.