9 days ago
Remote, Poland +2 moreSenior
Responsibilities
- Design and build auditable evaluation environments for frontier models covering reasoning, multimodal, long-context, and agentic systems.
- Research novel evaluation methodologies for emerging model families and capability domains, including low-precision numerics, multi-turn agentic tasks, and code generation.
- Build and operate benchmark environments, evaluation pipelines, regression CI systems, and statistical analysis tooling.
- Partner with research, training, product, and customer teams to turn evaluation signals into release, training, and competitive-positioning decisions.
Requirements
- BS, MS, or PhD in Computer Science, Machine Learning, Statistics, or a related field.
- 6+ years of hands-on experience with LLMs and evaluations for large language models or multimodal AI systems.
- Experience with agentic, multi-turn, or reasoning-heavy evaluation settings.
- Strong foundations in experimental design, significance testing, regression analysis, and large-scale benchmark analysis.
- Proven experience building evaluation infrastructure, pipelines, benchmark harnesses, and reproducible CI systems.
- Clear communication skills for translating quantitative evaluation results to researchers, product teams, and senior leadership.
- Preferred experience with open-source evaluation frameworks, agentic system evaluations, benchmark or methodology research, low-precision inference, quantization, HPC/Slurm clusters, MLflow, and W&B.
Tech Stack
MLflow
Categories
AI ResearchTesting
About Nvidia
Nvidia designs and sells GPUs and accelerated computing platforms for data centers, AI/ML, graphics, gaming, and automotive, monetizing through hardware, software platforms (CUDA, AI frameworks), and systems like DGX and networking. Customers include cloud providers, enterprises, researchers, and OEMs. Founded in 1993 and headquartered in Santa Clara, it is a public company traded on NASDAQ under NVDA.