1 month ago
Base Salary
$102k - $261k/yr
Responsibilities
- Design, implement, and operate offensive cyber-capability evaluations for frontier, preview, production, and open-weight AI models.
- Build and maintain evaluation tasks covering vulnerability analysis, exploit development, application and system exploitation, attack-path reasoning, post-exploitation, and multi-step offensive workflows.
- Define success criteria and baselines, execute controlled experiments, collect telemetry, and document results.
- Analyze model trajectories and investigate genuine capabilities, task leakage, environmental flaws, scoring errors, and other evaluation artifacts.
- Develop and maintain cyber ranges, vulnerable applications, exploit-development targets, and evaluation harnesses.
- Apply sandboxing, secrets management, telemetry, access controls, and containment for autonomous AI system testing.
- Partner with red-team operators, researchers, engineers, and Responsible AI stakeholders and communicate findings through reports, documentation, and briefings.
Requirements
- Bachelor's degree in Statistics, Mathematics, Computer Science, Computer Security, or a related field plus 2+ years of relevant experience, or a master's degree plus 1+ year of relevant experience, or equivalent experience.
- Relevant experience may include software development lifecycle, large-scale computing, threat analysis or modeling, cybersecurity, vulnerability research, or anomaly detection.
- Ability to meet Microsoft, customer, and government security-screening requirements, including export-control and citizenship verification requirements.
- Programming ability, particularly in Python, with experience building automation, scripts, or security tooling.
- Familiarity with large language models, generative AI systems, coding models, AI agents, model APIs, or model-evaluation frameworks.
- Experience or exposure to security labs, cyber ranges, capture-the-flag environments, vulnerable applications, exploit development, vulnerability research, penetration testing, application security, or red teaming.
- Experience designing controlled experiments, defining success criteria, analyzing results, and documenting technical findings.
- Familiarity with PyRIT, adversarial-testing tools, containers, sandboxed execution, or cloud-based test infrastructure.
- Familiarity with evaluation risks including contamination, task saturation, unreliable scoring, weak baselines, environmental leakage, and limited reproducibility.
About Microsoft
Microsoft develops operating systems, productivity software, cloud services, developer tools, and consumer devices for individuals, enterprises, and governments. Its main products include Windows, Microsoft 365, Azure, Visual Studio/GitHub, Xbox, and LinkedIn; revenue comes from software subscriptions and licenses, cloud consumption, hardware sales, and advertising. Founded in 1975 and headquartered in Redmond, Washington, Microsoft is a public company traded on Nasdaq.