3 days ago
London, United KingdomSenior
Responsibilities
- Research, experiment with, and prototype approaches for analyzing model behavior and failure patterns.
- Conduct empirical investigations into model outputs, agent trajectories, and error patterns to improve benchmark design and evaluation trustworthiness.
- Implement, test, and refine benchmark suites for complex agentic reasoning across code, math, and specialized domains.
- Develop and maintain pipelines that convert benchmark and sandbox execution logs, agent traces, and runtime artifacts into evaluation datasets and training data.
- Collaborate with research scientists, senior engineers, and cross-functional teams to translate research insights into evaluation infrastructure.
Requirements
- Bachelor’s degree or equivalent practical experience.
- At least 2 years of software development experience in one or more programming languages, or 1 year of experience with an advanced degree.
- At least 1 year of end-to-end machine learning experience, including model deployment, model evaluation, data processing, or debugging.
- Experience with generative AI techniques or concepts, including LLMs, multimodal systems, large vision models, language modeling, or computer vision.
- Experience in generative AI and Python.
- Preferred qualifications include a master’s degree or PhD in computer science or a related technical field.
- Preferred experience includes data structures and algorithms, accessible technology development, and data analysis.
Tech Stack
Google CloudPython
Categories
AI Research
About Google
Google builds consumer and enterprise software and services including Search, Android, YouTube, Chrome, Maps, Gmail, and Google Cloud. Its business model centers on digital advertising and paid cloud, software, and hardware offerings (e.g., Pixel and Nest) for consumers, developers, and organizations. Founded in 1998 and headquartered in Mountain View, California, Google operates globally as a subsidiary of Alphabet Inc.