AI Systems Engineer - DevOps& Observability - Senior
Ernst and Young1 month ago
Base Salary
$107k - $201k/yr
Responsibilities
- Build and operate CI/CD/CV pipelines for AI services, agents, and runtime components, including automated testing, continuous verification, release, and rollback.
- Operate production model-serving and inference systems on GPU infrastructure across cloud, on-premises, edge, and air-gapped environments.
- Own AI asset governance and discovery, including registries, experiment tracking, model metadata, vulnerability scanning, lineage, and license management.
- Manage quotas, rate limits, cost attribution, and utilization so AI consumption is bounded and attributable by tenant and workload.
- Own metrics, logs, traces, dashboards, LLM debugging and evaluation, SLA alerting, and OpenTelemetry collection and routing.
- Use telemetry, evaluation, and cost signals to drive deployment decisions, progressive rollout, and automated rollback.
- Define ownership boundaries and consumption contracts with platform, trust, and data teams.
Requirements
- At least 8 years of experience in DevOps, MLOps, platform, or observability engineering with hands-on production ownership of AI or high-throughput services.
- Strong hands-on experience with CI/CD/CV pipelines, GitOps tooling, automated build and release processes, and rollback.
- Hands-on expertise operating Ray Serve, vLLM, Triton, or NIM on GPU infrastructure.
- Strong experience with Prometheus, Grafana, Loki, Tempo or Jaeger, and OpenTelemetry.
- Experience with Envoy or an equivalent API gateway and request-routing technology, including streaming responses.
- Experience with OpenCost, Kubecost, or equivalent FinOps tooling, including quota and rate-limit enforcement.
- Familiarity with Harbor, MLflow, Trivy/SBOM scanning, artifact registries, supply-chain controls, lineage, and license tracking.
- Experience operating AI or service infrastructure under compliance, security, or regulatory constraints.
- Experience with LangSmith or Langfuse, sandboxed execution using gVisor or Firecracker, DCGM GPU telemetry, OpenLineage, and multi-tenant cost attribution is preferred.
- Strong communication skills and the ability to explain runtime, cost, and observability tradeoffs to engineers, architects, and leadership.
- A bachelor’s or master’s degree in computer science or a related technical field is preferred.
Benefits
- Base salary ranges from $106,900 to $176,500 nationally and from $128,400 to $200,600 in New York City Metro Area, Washington State, and California excluding Sacramento.
- Medical and dental coverage, pension and 401(k) plans, paid time off, EY paid holidays, winter and summer breaks, personal and family care leave, and other leave options.
- Hybrid, flexible work model; external client-serving employees are generally expected to work in person 40–60% of the time during an engagement, project, or year.
- Flexible vacation policy and ongoing application acceptance.
Tech Stack
Categories
About Ernst and Young
Ernst & Young (EY) provides audit/assurance, tax, consulting, strategy and transactions services to enterprises, financial institutions, and public‑sector clients. Structured as a global network of partner‑owned member firms, it sells professional services on a fee basis, including a dedicated Financial Services Organization for banking, insurance, and capital markets. Headquartered in London, EY was formed in 1989 from the merger of Ernst & Whinney and Arthur Young, and operates in 150+ countries.