5 hours ago
London, United KingdomSenior
Responsibilities
- Own the AI layer and backend integration of production systems for public sector clients.
- Build production agents with tool calling, guardrails, and observability.
- Design and adapt evaluations to identify regressions and measure client-specific risks.
- Apply context engineering, retrieval-augmented generation, model adaptation, and inference optimization.
- Investigate production regressions, lead root-cause analysis, and implement preventative changes.
- Lead workstream planning and manage client scope, risk, delivery, and expectations.
- Create reusable tooling, evaluation components, and guidance for other engineers.
- Mentor engineers through code review, pairing, and delegated ownership.
Requirements
- Backend engineering experience with a track record of taking AI/ML systems into production and maintaining their reliability.
- End-to-end ownership of production-system AI behavior rather than responsibility for only a team-scoped component.
- Hands-on experience designing evaluations connected to real-world outcomes and regression detection.
- Ability to work directly with clients and explain model behavior and trade-offs to non-technical stakeholders.
- Practical experience handling regulated or sensitive data and designing for auditability.
- Sovereign or on-premise deployment experience is valuable.
- Experience leading root-cause analysis on production AI or ML systems.
Categories
About Scale AI
Scale AI builds data annotation services and AI development tools for enterprises and government agencies, sold as a platform and managed services. Its products include the Scale Generative AI Platform for building and evaluating agents and the Data Engine for collecting, curating, and labeling training data, including RLHF and model evaluation. Founded in 2016 and headquartered in San Francisco, the company is privately held and works across domains from computer vision to LLM applications.