6 hours ago
Base Salary
$207k - $300k/yr
Responsibilities
- Construct quantitative benchmarks and automated evaluation frameworks, including LLM-as-a-judge systems, to measure agent reasoning, planning, and tool use.
- Develop and productionize agent evaluation-set curation workflows and agentic data pipelines.
- Build a self-service platform for prototyping and testing LLM-based classifiers and annotations against historical trajectory data.
- Establish metric lifecycle management with sandbox environments and high-confidence calibration experiments.
- Productionize tools that identify the largest contributors to metric changes across task types and user cohorts.
Requirements
- Bachelor’s degree in Computer Science or Engineering, or equivalent practical experience.
- Experience with Large Language Models, NLP, or Generative AI.
- MBA or master’s degree in Computer Science is preferred.
Categories
AI ApplicationsData Engineering
About Google
A problem isn't truly solved until it's solved for all. Googlers build products that help create opportunities for everyone, whether down the street or across the globe. Bring your insight, imagination and a healthy disregard for the impossible. Bring everything that makes you unique. Together, we can build for everyone. Check out our career opportunities at goo.gle/3DLEokh