Senior Staff Software Engineer - AI/ML Inference Engineering
ServiceNow1 day ago
Base Salary
$201k - $352k/yr
Responsibilities
- Lead the architecture, optimization, and operational excellence of the AI/ML inference platform.
- Own the full inference lifecycle, including model onboarding and deployment of optimized inference engines across multiple backends.
- Design and build an intelligent gateway using Envoy and ext_proc for access control, observability, and advanced request routing.
- Deploy, scale, and operate production services in Kubernetes using Helm-based deployments.
- Own metrics, logs, tracing, and customer-impact-focused alerting for platform services.
- Debug and resolve production incidents, participate in on-call, conduct root-cause analysis, and operate against defined service-level objectives.
Requirements
- 8+ years of backend software engineering experience and 4+ years designing and operating distributed systems.
- Bachelor’s degree in computer science, software engineering, or a closely related technical field; equivalent practical experience may be considered, and a master’s degree may offset up to one year of experience.
- Expertise in Go or Python, object-oriented programming, design patterns, efficient algorithms, and modular software development.
- Experience with model serving and runtimes such as vLLM, TensorRT-LLM, or Triton Inference Server.
- Knowledge of core AI/ML techniques, LLM APIs or ML-serving systems, unit testing, profiling, and code tuning.
- Experience building products using challenging algorithms and operating reliable production systems.
- Ability to use and critically review AI coding assistants such as GitHub Copilot or Cursor, including attention to security anti-patterns, edge cases, debugging, trace analysis, and prompt data privacy.
- Experience with Kubernetes production environments, Helm, repeatable rollback-safe releases, observability, incident response, and service-level objectives is desired.
Benefits
- Base pay is listed at $201,300–$352,300, with equity when applicable, variable or incentive compensation, and benefits.
- Health plans including flexible spending accounts, a 401(k) plan with company match, ESPP, matching donations, flexible time away, and family leave programs are offered.
- The role has a flexible work persona within ServiceNow’s distributed work model.
- This is a regular employee position in the North America and Canada region.
Tech Stack
Categories
About ServiceNow
ServiceNow builds a cloud platform for enterprise digital workflows, covering IT service management, customer service, HR service delivery, security operations, and operations management, plus tools for custom app development. It sells subscription SaaS to large organizations and public-sector agencies to automate processes and connect data across systems. Founded in 2004 and headquartered in Santa Clara, California, ServiceNow is a public company listed on the NYSE under the ticker NOW.