AI Systems Engineer - DevOps& Observability Manager
Ernst and Young13 days ago
Base Salary
$126k - $230k/yr
Responsibilities
- Build and operate CI/CD/CV pipelines for AI services, agents, and runtime components, including automated build, testing, continuous verification, release, and rollback.
- Own AI asset governance and discovery across service registries, experiment tracking, model metadata, upstream registries, CVE/SBOM scanning, lineage contracts, and license management.
- Manage quotas, rate limits, cost attribution, and utilization so AI execution remains economically bounded per tenant and engagement.
- Own metrics, logs, traces, dashboards, LLM debugging and evaluation, SLA monitoring, and alert notifications.
- Operate the OpenTelemetry collection layer with multi-tenant receivers, exporters, queues, GPU telemetry, batching, and dynamic filtering.
- Use telemetry, evaluation, and cost signals to drive deployment decisions, progressive rollout, and automated rollback.
- Ensure cost and telemetry are identity-stamped and attributable to individual tenants and workloads across cloud, on-premises, edge, and air-gapped environments.
Requirements
- Bachelor’s or master’s degree in Computer Science or a related technical field.
- At least 8 years of experience in DevOps, MLOps, platform, or observability engineering with hands-on production ownership of AI or high-throughput services.
- Strong DevOps experience with CI/CD/CV pipelines, GitOps, continuous verification, progressive release, and automated rollback.
- Hands-on experience operating Ray Serve, vLLM, Triton, or NIM on GPU infrastructure.
- Strong experience with Prometheus, Grafana, Loki, Tempo or Jaeger, and OpenTelemetry.
- Experience with API gateways and request routing such as Envoy, including streaming responses.
- Experience with FinOps tooling such as OpenCost or Kubecost and quota/rate-limit enforcement.
- Familiarity with Harbor, MLflow, Trivy/SBOM, artifact registries, supply-chain scanning, license tracking, and lineage governance.
- Proven experience operating AI or service infrastructure under compliance, security, or regulatory constraints.
- Ability to define ownership boundaries and consumption contracts with platform, trust, and data teams.
- Preferred experience with LangSmith, Langfuse, sandboxed execution using gVisor or Firecracker, DCGM GPU telemetry, OpenLineage, AI license management, per-tenant cost attribution, SLA/alerting, and regulated delivery environments.
Benefits
- Base salary is $125,500 to $230,200 across US geographic locations, with higher ranges of $150,700 to $261,600 for New York City Metro Area, Washington State, and California excluding Sacramento.
- Medical and dental coverage, pension and 401(k) plans, paid time off, EY-paid holidays, winter and summer breaks, personal and family care leave, and other leaves of absence.
- Flexible vacation policy and a flexible, team-led and leader-enabled hybrid work model, with most external client-serving roles expected to work in person 40–60% of the time during an engagement, project, or year.
- Applications are accepted on an ongoing basis.
Tech Stack
Categories
About Ernst and Young
Ernst & Young (EY) provides audit/assurance, tax, consulting, strategy and transactions services to enterprises, financial institutions, and public‑sector clients. Structured as a global network of partner‑owned member firms, it sells professional services on a fee basis, including a dedicated Financial Services Organization for banking, insurance, and capital markets. Headquartered in London, EY was formed in 1989 from the merger of Ernst & Whinney and Arthur Young, and operates in 150+ countries.