4 months ago
Taguig City, PhilippinesMid Level / Senior
Responsibilities
- Build and grow the automated evaluation suite for AI agents.
- Design and automate adversarial test scenarios for continuous evaluation.
- Own the 'break it on purpose' testing process for new agents.
- Collaborate with the Data Steward on data sensitivity classification.
- Establish automated thresholds for agent capability evaluations.
- Maintain a guardrail and negative-test catalog across the platform.
- Produce clear, audit-ready evidence of evaluation results.
- Track drift over time across the entire fleet of agents.
Requirements
- At least 4 years in software quality or security testing.
- 1-2 years of experience evaluating or red-teaming LLM-based systems.
- Strong Python skills for building evaluation harnesses and test scripts.
- Working knowledge of REST APIs and event-driven patterns.
- Experience in building automated test infrastructure integrated into CI/CD.
- Familiarity with adversarial testing tools and standards like OWASP LLM Top 10.
- Practical understanding of prompt injection and jailbreaking failure modes.
- Willingness to block releases that do not meet quality standards.
About Netskope
Netskope builds cloud security and networking software for enterprises, bundling CASB, SWG, ZTNA, DLP, and an enterprise browser in the Netskope One Security Service Edge/SASE platform. It sells subscriptions and services to security and networking teams to protect data and manage access across SaaS, web, private apps, and public cloud. Founded in 2012 and headquartered in Santa Clara, California, Netskope says it serves thousands of customers, including dozens of Fortune 100 companies.