Netflix

Software Engineer 5, AI Observability

Netflix
Apply
2 days ago
Remote, United States or USA (city not specified)Staff+
H1B sponsor

Base Salary

$500k - $860k/yr

Responsibilities

  • Build observability frameworks and platform capabilities for ML and GenAI systems across online inference, batch scoring, feature pipelines, and agent orchestration.
  • Create reusable primitives for monitoring model performance, data quality, drift, degradation, latency, cost, and reliability.
  • Build evaluation frameworks for LLM and agentic systems covering response quality, grounding, hallucination, task success, tool use, trajectory correctness, LLM-as-a-judge, and human-in-the-loop scoring.
  • Lead build-versus-buy evaluations and own SDKs, connectors, and APIs for integrating vendor observability and evaluation platforms.
  • Develop reusable libraries, SDKs, templates, dashboards, alerting, and SLO/SLI building blocks that enable observability by default.
  • Partner with engineering, product, machine learning, data, and infrastructure teams to turn platform needs into reusable capabilities.

Requirements

  • Experience in software, AI/ML, or platform engineering with production observability, monitoring, or ML/LLM evaluation experience.
  • A proven track record designing standards, libraries, SDKs, or frameworks adopted by other engineering teams.
  • Strong coding skills in Python and at least one of Java, Go, or Scala, plus experience building production services.
  • Practical experience operating ML models in production through online serving or batch processing, including model drift, performance, and data quality.
  • Hands-on experience with modern observability stacks such as Prometheus/Grafana, Datadog, OpenTelemetry, ELK/OpenSearch, or Jaeger/Tempo.
  • Understanding of distributed systems, microservices, and at least one major cloud platform: AWS, GCP, or Azure.
  • Experience with vendor integration and VPC deployment.
  • Experience with ML/LLM observability and evaluation tools such as Arize, Braintrust, LangFuse, Weights & Biases, Galileo, Vertex AI Model Monitoring, or SageMaker Model Monitor is preferred.
  • Experience building or shipping LLM/GenAI applications and evaluating prompts, results, metrics, LLM-as-a-judge workflows, and human-in-the-loop review is preferred.
  • Experience evaluating agentic systems, including tool use, multi-step reasoning, and trajectory or task-success measurement, is preferred.
  • Ability to collaborate cross-functionally and communicate clearly about system behavior, quality, and risk.
  • Uses AI tools as a core part of the engineering workflow for design, development, testing, and code review.

Benefits

  • Comprehensive health plans, mental health support, disability programs, health savings and flexible spending accounts, family-forming benefits, and life and serious injury benefits.
  • 401(k) retirement plan with employer match and a stock option program.
  • Paid leave of absence programs.
  • Full-time salaried employees receive flexible time off immediately; full-time hourly employees accrue 35 days annually for vacation, holidays, and sick time.
  • Annual compensation is structured as salary and stock options with no bonuses; the listed compensation range is location-dependent.

Tech Stack

AWSAzureDatadogGoGoogle Cloud PlatformGrafanaJavaPrometheusPythonScala
Netflix

About Netflix

10,000+ employees

Netflix builds and operates a global streaming service for TV series, films, games, and live programming, and produces original content through its in-house studio. It serves consumers in 190+ countries via subscription and ad-supported plans on internet-connected devices. Founded in 1997 and headquartered in Los Gatos, California, Netflix is a public company traded on NASDAQ (NFLX).

Contact me