2 days ago
Base Salary
$174k - $252k/yr
Responsibilities
- Design, build, and maintain reusable evaluation infrastructure blocks such as streaming evaluation, autorater pipelines, and metric aggregation.
- Improve the scalability, reliability, and efficiency of large-scale evaluation workflows across compute, storage, and serving backends.
- Onboard product teams, debug their issues, and convert recurring needs into self-service features.
- Build secure per-customer data isolation and access control for evaluation data and results.
- Help teams transition from offline evaluations to online evaluations, live experiments, and production.
Requirements
- Bachelor's degree or equivalent practical experience.
- 5 years of experience coding in Python.
- 3 years of experience developing large-scale infrastructure, distributed systems or networks, or working with compute technologies, storage, or hardware architecture.
- 3 years of experience working with Generative AI agents.
- 2 years of experience with multithreading.
- Preferred: Master's degree or PhD in Computer Science or a related technical field.
- Preferred: 2 years of experience with large-scale data processing, storage systems, or workflow orchestration.
- Preferred: 2 years of experience with at least two of distributed and parallel systems, machine learning, information retrieval, natural language processing, Unix/Linux environments, or security and access control.
- Preferred: Experience with ML infrastructure, model evaluation, or LLM and GenAI workflows such as autoraters, metrics, or experimentation platforms.
- Preferred: Experience developing in Python and C++.
Benefits
- 15% bonus target, equity, and benefits are offered in addition to the stated base salary.
- The role is based in the US; no remote or office arrangement is stated.
Categories
About Google
Google builds consumer and enterprise software and services including Search, Android, YouTube, Chrome, Maps, Gmail, and Google Cloud. Its business model centers on digital advertising and paid cloud, software, and hardware offerings (e.g., Pixel and Nest) for consumers, developers, and organizations. Founded in 1998 and headquartered in Mountain View, California, Google operates globally as a subsidiary of Alphabet Inc.