Software Engineer, Evaluation Platform / Infra

2 hours ago

H1B Sponsor

Base Salary

$300k - $475k/yr

Responsibilities

Design, build, and maintain a platform for authoring, running, tracking, and analyzing model evaluations.
Develop evaluation libraries, distributed backend systems, data pipelines, APIs, and user-facing applications end to end.
Build abstractions for evaluation tasks, environments, graders, datasets, and model outputs.
Improve reproducibility and trust through versioning, provenance, observability, failure recovery, and quality controls.
Partner with researchers to convert bespoke workflows and bottlenecks into self-serve systems used across teams.
Collaborate with the Research Tooling team and other pre-training, post-training, and applied teams.

Bachelor’s degree or equivalent practical experience in computer science, engineering, machine learning, or a related field.
At least two years of post-graduate software engineering or ML engineering experience, excluding internships.
Hands-on experience building or maintaining evaluations, benchmarks, graders, or model-quality systems for large language or multimodal models.
Strong software engineering fundamentals and experience building reliable, maintainable systems.
Proficiency in at least one backend programming language, with Python and Rust used primarily.
Experience with databases, data pipelines, distributed systems, or other data-intensive infrastructure.
Ability to work across the stack and own projects from problem discovery through deployment and operation.
Experience collaborating with cross-functional partners and subject-matter experts.
Preferred experience includes building frameworks, SDKs, developer tools, distributed job execution, workflow orchestration, sandboxed environments, large-scale data processing, polished data-inspection interfaces, model evaluation, research collaboration, or end-to-end work on small teams.

11-50 employees