1 month ago
Remote, United StatesSenior
Responsibilities
- Develop and execute software and AI system test plans aligned with the OWASP AI Testing Guide and applicable NIST, ISO, IEEE, and NSA guidance.
- Evaluate AI systems for intended functionality, model robustness, data integrity, adversarial defense, performance, usability, reliability, security, maintainability, safety, and overall quality.
- Lead testing strategy and quality engineering transformation efforts for large-scale programs, including requirements traceability throughout the software development lifecycle.
- Plan and execute system integration, user acceptance, performance, load, regression, white-box, black-box, and coverage testing.
- Create defect and remediation records, capture test evidence, document reproducible defects with severity ratings, and report findings to stakeholders and senior management.
- Evaluate LLM-based applications, including chatbots, copilots, and retrieval-augmented generation systems, through prompt-response, hallucination, and edge-case testing.
- Implement AI-driven quality engineering solutions such as autonomous test generation, self-healing automation, defect prediction, and intelligent triage.
Requirements
- Bachelor's degree in computer science, engineering, or a related field.
- At least 10 years of experience in software or system testing and quality engineering, including enterprise-scale transformation programs.
- At least 4 years of hands-on experience defining testing strategy for large-scale programs and maintaining requirements traceability.
- Experience leading testing efforts across system integration, user acceptance, performance, and other testing phases.
- Hands-on expertise with agentic AI and generative AI frameworks such as LangChain, AutoGen, CrewAI, and Cursor.
- Experience with machine learning models, Python NLTK or similar natural language processing technology, and AI-driven quality engineering solutions.
- Expertise with test automation tools such as Selenium, Playwright, and Cypress.
- Experience with cloud, web, and mobile testing platforms; Sauce Labs and BrowserStack are preferred.
- Experience with CI/CD pipelines, DevOps integration, GitHub, cloud-native architectures, APIs, and microservices.
- Experience evaluating AI and LLM-based application features, including prompt-response validation, hallucination testing, and edge-case testing.
- Experience with AWS, Azure, or GCP in quality engineering or AI contexts.
- Understanding of responsible AI and testing governance principles, including bias detection, explainability, and model validation.
- Experience influencing AI adoption roadmaps, implementing testing best practices, and gaining executive support for process changes.
- Preferred qualifications include familiarity with JIRA, WCAG/accessibility, IT security, IT privacy, AI-assisted test tools, and professional software testing certifications.
Benefits
- Remote-friendly, part-time consulting engagement with occasional travel to customer sites or TestPros labs in Sterling, Virginia.
- Future projects are expected to begin in late 2026 or 2027 rather than immediately.
- Engagement may be structured as 1099 or Corp. to Corp.
- Occasional on-site or hybrid work may be required based on project needs.
Tech Stack
Categories
