4 months ago
Base Salary
$216k - $270k/yr
Responsibilities
- Design and build platform capabilities for deploying, monitoring, and evaluating production agentic applications.
- Build APIs and data pipelines that capture agent telemetry, evaluation signals, and performance metrics at scale.
- Partner with ML engineers and applied researchers to turn evaluation and improvement workflows into reliable platform capabilities.
- Own the reliability, scalability, and observability of platform components used by enterprise and government customers.
- Translate customer and deployment requirements into platform features with product, forward deployed engineering, and customer-facing teams.
- Deliver features end-to-end from system design and implementation through debugging, testing, and production operation.
- Participate in experimentation and provide substantive design and code feedback while mentoring other engineers.
Requirements
- 4+ years of professional software engineering experience with backend or distributed systems, APIs, and data pipeline design.
- Hands-on experience building production software for ML/LLM-powered products or platforms, including evaluation, observability, experimentation, agent runtime, model-serving-adjacent, or telemetry systems.
- Working knowledge of production ML/LLM behavior, evaluation signals, failure modes, prompt and tool-calling workflows, experiment results, and data quality issues.
- Experience partnering with ML engineers or applied researchers to productionize prototypes, evaluation loops, or model-improvement workflows without owning model training or research direction.
- Experience building infrastructure, internal platforms, developer tools, or other systems used by engineering teams.
- Demonstrated ownership of features or components from design through production.
- Ability to work independently and cross-functionally in an ambiguous, fast-changing environment.
- Preferred: deep experience with ML/LLM observability, monitoring, or evaluation systems; agent architectures; MLOps; feature stores; model serving; experiment infrastructure; regulated or enterprise contexts; and technical design review or mentoring at a senior/staff level.
Benefits
- Base salary, equity, comprehensive health, dental, and vision coverage, retirement benefits, learning and development stipend, generous PTO, and potentially a commuter stipend.
- Full-time position with a listed base salary range for San Francisco, New York, and Seattle.
Categories
BackendData Engineering
About Scale AI
Scale AI builds data annotation services and AI development tools for enterprises and government agencies, sold as a platform and managed services. Its products include the Scale Generative AI Platform for building and evaluating agents and the Data Engine for collecting, curating, and labeling training data, including RLHF and model evaluation. Founded in 2016 and headquartered in San Francisco, the company is privately held and works across domains from computer vision to LLM applications.