3 hours ago
Remote, WorldwideMid Level / Senior
Base Salary
$110k - $160k/yr
Responsibilities
- Design and build sandboxed environments where AI models can safely execute code, use tools, interact with services, and complete complex tasks.
- Build backend services and infrastructure for large-scale, repeatable AI and agentic evaluations.
- Develop agent scaffolding and evaluation harnesses with tool-use loops, context management, retries, state management, token budgets, and multi-agent workflows.
- Provision and orchestrate isolated environments using Docker, Kubernetes, virtual machines, and cloud infrastructure.
- Design secure networking, permissions, secrets, credentials, and resource isolation for model-driven environments.
- Develop APIs, internal tools, and automation for creating and running evaluations.
- Improve reliability and reproducibility through logging, observability, snapshotting, debugging tools, and automated testing.
- Build systems that run thousands of evaluation tasks reliably and capture relevant artifacts and telemetry.
- Partner with analysts, red teamers, and domain experts to translate evaluation concepts into technical systems.
- Investigate failures across the evaluation stack and distinguish model limitations from infrastructure, harness, or environment failures.
Requirements
- 3–5+ years of professional software engineering experience, particularly in backend, infrastructure, platform, SRE, or distributed systems engineering.
- Strong programming skills in Python and experience building production-quality software.
- Experience designing and operating backend services, APIs, or distributed systems.
- Hands-on experience with Docker, Kubernetes, virtual machines, or other container/orchestration technologies.
- Experience with AWS, GCP, or similar cloud infrastructure.
- Strong understanding of Linux systems, networking, authentication, permissions, and infrastructure security.
- Experience with infrastructure-as-code or automation tools such as Terraform.
- Strong debugging skills across application, infrastructure, and networking layers, especially in agentic loops.
- Ability to build reproducible, observable, scalable, and secure systems.
- Interest in AI systems, agentic workflows, AI security, or model evaluations; prior professional AI experience is helpful but not required.
- Experience with developer platforms, test infrastructure, sandboxes, ephemeral compute environments, agent frameworks, LLM APIs, tool-calling systems, distributed task execution, queues, workflow orchestration, or large-scale automated testing is a plus.
- Familiarity with AI safety, adversarial testing, model evaluations, autonomous-agent systems, Model Context Protocol, agent benchmarks, and AI-agent security risks is a plus.
Benefits
- Professional development support for conferences, continuing education, or leadership training.
- Fully remote work environment for U.S.-based employees.
- Comprehensive health, dental, and vision coverage.
- Generous PTO and paid holiday schedule.
- 401(k) plan.
- Performance-based annual bonus.
About 10a Labs
10a Labs is the safety and threat-intelligence layer trusted by frontier AI labs, AI unicorns, Fortune 10 companies, and leading global technology platforms. Our adversarial red teaming, model evaluations, and intelligence collection enable engineering, safety, and security teams to stay ahead of evolving threats and deploy AI systems safely.