Full stack engineer
Judgment Labs3 months ago
Responsibilities
- Build the Judgment Agent to run large-scale, parallel investigations across production traces and merge findings into unified results.
- Build verification infrastructure including hosted simulated environments, trajectory replay, and monitors for unintended agent behavior changes.
- Design interfaces for understanding agent traces, tool calls, decisions, failures, and investigations.
- Create swarm user experiences for monitoring, redirecting, and consuming parallel investigations.
- Build workflows that turn production trajectories into datasets, judges, and regression checks.
- Develop platform functionality for workspaces, roles, permissions, billing, usage, and limits.
- Build an SDK and terminal-first experience that enables Claude Code, Codex, and OpenCode sessions to use Judgment as a subagent.
- Own product problems end to end by speaking with customers, defining solutions, building them, and iterating.
Requirements
- Experience building and scaling end-to-end production systems across the data layer and user interface.
- Strong technical problem-solving skills in fast-changing and ambiguous environments.
- A builder and tinkerer mindset with high agency and creative approaches to overcoming obstacles.
- Hands-on experience building with LLMs or agents, or the ability to develop that experience quickly.
- Comfort working directly with customers to understand their needs and solve real-world problems.
- Excellent communication skills across technical and non-technical audiences.
Benefits
- Full-time, on-site role in San Francisco.
Categories
About Judgment Labs
Judgment Labs builds learning and evaluation infrastructure for AI agents, helping engineering teams monitor behavior, surface failure modes, and improve performance from real production traces. Its platform and SDK provide continuous testing, rubrics, and large-scale investigations across tool errors, regressions, and drift. Founded in 2025 and headquartered in San Francisco, the privately held company sells B2B software for teams deploying long‑trajectory agents in development and production environments.