2 days ago
Base Salary
$500k - $860k/yr
Responsibilities
- Build observability frameworks and platform capabilities for ML and GenAI systems across online inference, batch scoring, feature pipelines, and agent orchestration.
- Create reusable primitives for monitoring model performance, data quality, drift, degradation, latency, cost, and reliability.
- Build evaluation frameworks for LLM and agentic systems covering response quality, grounding, hallucination, task success, tool use, trajectory correctness, LLM-as-a-judge, and human-in-the-loop scoring.
- Lead build-versus-buy evaluations and own SDKs, connectors, and APIs for integrating vendor observability and evaluation platforms.
- Develop reusable libraries, SDKs, templates, dashboards, alerting, and SLO/SLI building blocks that enable observability by default.
- Partner with engineering, product, machine learning, data, and infrastructure teams to turn platform needs into reusable capabilities.
Requirements
- Experience in software, AI/ML, or platform engineering with production observability, monitoring, or ML/LLM evaluation experience.
- A proven track record designing standards, libraries, SDKs, or frameworks adopted by other engineering teams.
- Strong coding skills in Python and at least one of Java, Go, or Scala, plus experience building production services.
- Practical experience operating ML models in production through online serving or batch processing, including model drift, performance, and data quality.
- Hands-on experience with modern observability stacks such as Prometheus/Grafana, Datadog, OpenTelemetry, ELK/OpenSearch, or Jaeger/Tempo.
- Understanding of distributed systems, microservices, and at least one major cloud platform: AWS, GCP, or Azure.
- Experience with vendor integration and VPC deployment.
- Experience with ML/LLM observability and evaluation tools such as Arize, Braintrust, LangFuse, Weights & Biases, Galileo, Vertex AI Model Monitoring, or SageMaker Model Monitor is preferred.
- Experience building or shipping LLM/GenAI applications and evaluating prompts, results, metrics, LLM-as-a-judge workflows, and human-in-the-loop review is preferred.
- Experience evaluating agentic systems, including tool use, multi-step reasoning, and trajectory or task-success measurement, is preferred.
- Ability to collaborate cross-functionally and communicate clearly about system behavior, quality, and risk.
- Uses AI tools as a core part of the engineering workflow for design, development, testing, and code review.
Benefits
- Comprehensive health plans, mental health support, disability programs, health savings and flexible spending accounts, family-forming benefits, and life and serious injury benefits.
- 401(k) retirement plan with employer match and a stock option program.
- Paid leave of absence programs.
- Full-time salaried employees receive flexible time off immediately; full-time hourly employees accrue 35 days annually for vacation, holidays, and sick time.
- Annual compensation is structured as salary and stock options with no bonuses; the listed compensation range is location-dependent.
Categories
About Netflix
Netflix builds and operates a global streaming service for TV series, films, games, and live programming, and produces original content through its in-house studio. It serves consumers in 190+ countries via subscription and ad-supported plans on internet-connected devices. Founded in 1997 and headquartered in Los Gatos, California, Netflix is a public company traded on NASDAQ (NFLX).
