8 hours ago
Base Salary
$174k - $252k/yr
Responsibilities
- Build human-powered and Large Language Model-powered automated evaluation systems for assessing model performance.
- Define metrics for grounding, coherence, safety, helpfulness, and related model-quality dimensions.
- Use platforms and tools to run evaluations across multiple models and datasets.
- Provide actionable evaluation insights to improve model quality in collaboration with research and cross-functional teams.
- Create tools and systems that make model evaluation more efficient and effective.
Requirements
- Bachelor’s degree or equivalent practical experience.
- Five years of software development experience in one or more programming languages.
- Three years of experience testing, maintaining, or launching software products.
- One year of experience with software design and architecture.
- Preferred: master’s degree or PhD in Computer Science or a related technical field.
- Preferred: five years of experience in speech/audio, reinforcement learning, ML infrastructure, or another ML specialization.
- Preferred: five years of experience with ML design and ML infrastructure, including model deployment, model evaluation, data processing, debugging, or fine-tuning.
- Preferred: three years of experience working in a complex, matrixed organization on cross-functional or cross-business projects.
- Preferred: experience integrating generative AI tools or Large Language Model interfaces into workflows.
Benefits
- US base salary is $174,000–$252,000, plus a 15% bonus target, equity, and benefits.
- Role contributes to Google Pics, an AI-powered visual editor integrated across Google Workspace.
- Opportunities to work across projects and teams in Google’s software engineering organization.
Categories
About Google
Google builds consumer and enterprise software and services including Search, Android, YouTube, Chrome, Maps, Gmail, and Google Cloud. Its business model centers on digital advertising and paid cloud, software, and hardware offerings (e.g., Pixel and Nest) for consumers, developers, and organizations. Founded in 1998 and headquartered in Mountain View, California, Google operates globally as a subsidiary of Alphabet Inc.