1 day ago
Base Salary
$184k - $357k/yr
Responsibilities
- Design and maintain safety benchmarks, test cases, datasets, scoring rubrics, metrics, and evaluation protocols for large language and multimodal models.
- Build reusable Python pipelines with versioned datasets, model configurations, and experiment-tracking workflows.
- Develop automated red-teaming tools, validate automated scoring with human review, and expand evaluation coverage.
- Investigate model failures, document actionable findings, and create regression tests for model changes.
- Partner with researchers and platform engineers to store, aggregate, and compare evaluation results.
- Communicate evaluation findings, limitations, tradeoffs, and emerging risks to researchers, engineering partners, and decision-makers.
Requirements
- Bachelor’s degree or equivalent experience and 6+ years in machine learning engineering or evaluation.
- Experience developing benchmarks, metrics, statistically sound comparisons, baselines, uncertainty analyses, and testable ML evaluation experiments.
- Strong Python skills and experience building maintainable shared tools, data pipelines, and experiment-tracking workflows.
- Ability to assess dataset quality, identify limitations in metrics and scoring methods, and support conclusions with reproducible evidence.
- Ability to clearly explain findings and tradeoffs to researchers, engineering partners, and executives.
- Preferred qualifications include evaluating LLMs, multimodal models, or agentic systems; adversarial testing or automated red-teaming; contributions to public benchmarks or evaluation frameworks; research publications; human-aligned evaluation; and confidence-interval expertise.
Benefits
- Eligible for equity and benefits.
- Applications will be accepted at least until November 9, 2026.
Tech Stack
Categories
About Nvidia
Nvidia designs and sells GPUs and accelerated computing platforms for data centers, AI/ML, graphics, gaming, and automotive, monetizing through hardware, software platforms (CUDA, AI frameworks), and systems like DGX and networking. Customers include cloud providers, enterprises, researchers, and OEMs. Founded in 1993 and headquartered in Santa Clara, it is a public company traded on NASDAQ under NVDA.