Base Salary
$175k - $220k/yr
Responsibilities
- Develop reusable platforms for in-house and open-source LLM benchmarks.
- Ensure the correctness and performance of post-training and evaluation jobs.
- Improve APIs for managing ML workflows.
- Contribute to foundational infrastructure for model inference and training.
- Participate in the team’s on-call process to maintain service availability.
- Own projects end-to-end from requirements and scoping through design and implementation.
Requirements
- 4+ years of experience developing ML platforms.
- Strong fundamentals in machine learning, backend system design, infrastructure, and large-scale systems.
- Experience training and/or benchmarking LLMs.
- Experience with Python, Docker, Kubernetes, and infrastructure as code such as Terraform.
- Experience building, deploying, and monitoring complex microservice architectures is preferred.
- Experience with a cloud technology stack such as AWS or GCP is preferred.
- Ability to collaborate closely with researchers and diagnose both model performance and system failures.
Benefits
- Comprehensive health, dental, and vision coverage.
- Retirement benefits, learning and development stipend, and generous paid time off.
- May be eligible for a commuter stipend.
- Full-time position located in San Francisco, New York, or Seattle.
- Equity-based compensation may be available for eligible roles.
Categories
About Scale AI
Scale’s mission is to develop reliable AI systems for the world’s most important decisions. We provide the high-quality data and full-stack technologies that power the world’s leading models, and help enterprises and governments build, deploy, and oversee AI applications that deliver real impact. The Scale Generative AI Platform allows customers to build, evaluate, and control advanced AI agents and applications that continuously improve. The Scale Data Engine provides the technology to collect, curate, and annotate high-quality datasets. Through our Scale Labs, we test models with rigorous benchmarks and novel research to ensure breakthroughs translate into systems people can trust. Scale powers the most advanced LLMs and generative models in the world through RLHF, data generation and model evaluation. We work with industry leaders like Meta, Cisco, DLA Piper, Mayo Clinic, Time Inc., the Government of Qatar, and U.S. government agencies including the Army and Air Force.