6 hours ago
Base Salary
$207k - $300k/yr
Responsibilities
- Design and scale production developer agents, including trajectory planning, tool selection, and state recovery.
- Implement human-in-the-loop approval gates, intent disambiguation, and graceful handling mechanisms for safe action execution.
- Architect evaluation pipelines for multi-step trajectories, tool-call fidelity, intermediate states, and reasoning paths.
- Create closed-loop pipelines that convert execution failures and human corrections into golden datasets, prompt and tool-schema optimizations, and fine-tuning cycles.
- Curate end-to-end benchmarks for complex developer tasks and establish standardized KPIs for agent workflows.
- Lead architecture, evaluation, and quality engineering for Google’s developer agent ecosystem.
Requirements
- Bachelor’s degree or equivalent practical experience.
- At least 8 years of software development experience.
- At least 5 years of experience testing and launching software products and 3 years of experience with software design and architecture.
- At least 5 years of experience with ML design and ML infrastructure, including model deployment, model evaluation, data processing, debugging, or fine-tuning.
- Experience integrating generative AI tools or LLM interfaces into workflows.
- Preferred: Master’s degree or PhD in Engineering, Computer Science, or a related technical field.
- Preferred: 3 years of technical leadership experience, such as leading technical roadmaps, designing multi-system architectures, and guiding cross-functional engineering efforts.
- Preferred experience with AI/ML evaluations, LLM benchmarking frameworks, agentic quality measurement pipelines, prompt optimization, LLM fine-tuning, automated dataset generation, trajectory-based evaluation, and productionizing generative AI solutions or autonomous agent loops.
Benefits
- US base salary of $207000 - $300000, plus a 20% bonus target, equity, and benefits.
- Google benefits are available; additional work arrangement or logistics are not stated.
Categories
About Google
Google builds consumer and enterprise software and services including Search, Android, YouTube, Chrome, Maps, Gmail, and Google Cloud. Its business model centers on digital advertising and paid cloud, software, and hardware offerings (e.g., Pixel and Nest) for consumers, developers, and organizations. Founded in 1998 and headquartered in Mountain View, California, Google operates globally as a subsidiary of Alphabet Inc.