Software Engineer – AI Quality & Test Automation | Security - Python | Golang | Test Automation | AI/ML Testing | Performance Testing | Security Testing | LLM APIs | Agentic Workflows | LangChain | LangSmith | Contract Testing | Playwright (8 - 12 Years)
Cisco16 hours ago
Bengaluru, IndiaSenior / Staff+
Responsibilities
- Develop PQ foundations, benchmarking protocols, and evaluation frameworks for AI agents, workflows, and system processes.
- Build scalable, maintainable, and high-performance testing infrastructure using Python and Go.
- Implement testing methodologies, AI-based automation, proof-of-concepts, and validation paradigms.
- Execute LLM-as-judge validation, tenant isolation probes, behavioral regression suites, contract testing, and audit log validation.
- Collaborate on code and design reviews, validate technical hypotheses, and maintain reliable quality processes.
- Partner with engineering teams on security compliance and quality integration across the development lifecycle.
Requirements
- Bachelor’s degree plus 8+ years of experience in software development and test automation, including large-scale testing ecosystems.
- Programming experience with Python and Go and the ability to design scalable, maintainable, high-performance testing infrastructure.
- Experience with security compliance frameworks and modern performance testing methodologies.
- Experience using AI to build automation, including writing AI skills for test automation.
- Preferred experience with contract testing frameworks, Playwright, OCSF validation libraries, vacuum linter, Caldera, tenant isolation probes, and audit log validation.
- Preferred expertise with LLM APIs including Claude and Codex, prompt engineering, agentic workflows, LangChain, LangSmith, AI behavioral regression, and LLM-as-judge frameworks.
- Preferred experience with K6, Terra Virtual Machine (Security Suite), and JMeter.
About Cisco
Cisco is the worldwide technology leader that is revolutionizing the way organizations connect and protect in the AI era. For more than 40 years, Cisco has securely connected the world. With its industry leading AI-powered solutions and services, Cisco enables its customers, partners and communities to unlock innovation, enhance productivity and strengthen digital resilience. With purpose at its core, Cisco remains committed to creating a more connected and inclusive future for all.