GrepJob
Scale AI

Senior Software Engineer, Agent Oversight

Scale AI
Apply
about 6 hours ago
San Francisco, CA, USA or New York, NY, USASenior / Mid Level
H1B Sponsor

Base Salary

$216k - $270k/yr

Responsibilities

  • Design and build core platform capabilities for deploying and evaluating agentic applications.
  • Build reliable APIs and data pipelines to capture agent telemetry and performance metrics.
  • Collaborate with ML engineers to ensure reliable workflows for evaluation and improvement.
  • Own the reliability and scalability of platform components for enterprise and government customers.
  • Translate real-world deployment requirements into platform features through cross-functional collaboration.
  • Participate in high-velocity experimentation to validate platform capabilities.

Requirements

  • 4+ years of professional software engineering experience with a focus on backend/distributed systems.
  • Hands-on experience building production software for ML/LLM-powered products.
  • Working knowledge of LLM or ML systems in production, including evaluation signals and failure modes.
  • Experience partnering with ML engineers to develop reliable platform capabilities.
  • Track record of taking ownership of features from design through production.
  • Strong problem-solving skills and ability to work independently or in a team.

Benefits

  • Comprehensive health, dental, and vision coverage.
  • Retirement benefits.
  • Learning and development stipend.
  • Generous PTO.
  • Potential commuter stipend.

Categories

AI & MLBackendData Engineering