DE-NCLC-SM-Agenetic AI
Ernst and Young1 day ago
Bengaluru, IndiaStaff+
Responsibilities
- Own the end-to-end test strategy, test plans, acceptance-criteria coverage, and release-quality evidence for the agentic AI product.
- Build and maintain automated regression and integration tests using Playwright, Cypress, Selenium, API testing frameworks, Python, and Python testing libraries.
- Validate agent tool selection and sequencing, function-call inputs and outputs, orchestration paths, refusal and escalation behavior, and human-in-the-loop controls.
- Design positive, negative, ambiguous, adversarial, and out-of-scope scenarios for relevant acceptance criteria.
- Maintain evaluation datasets and regression suites for AI-generated outputs, focusing on correctness, consistency, factual grounding, and regression prevention.
- Validate agent execution logs and traceability across perception, reasoning, planning, execution, retries and corrections, and resulting outputs.
- Test security and access-control scenarios, including negative authorization cases and agent/tool boundaries, in coordination with Security and Compliance SMEs.
- Perform defect triage and root-cause analysis, track defects through closure, and provide quality and release-readiness reporting through scripts and dashboards.
- Support performance and regression testing and contribute evidence for system testing, UAT, and release gates.
Requirements
- Senior Manager-level experience with strong Python, automation, and agentic AI capabilities.
- Experience owning test strategy, automated regression and integration testing, acceptance-criteria coverage, and release-quality evidence.
- Experience testing AI-agent behavior, generated outputs, execution traces, orchestration, tool boundaries, and human-in-the-loop controls.
- Experience with Playwright, Cypress, Selenium, API testing frameworks, and Python testing libraries.
- Experience with security and access-control testing, defect triage, root-cause analysis, performance testing, UAT, and release-readiness reporting.
About Ernst and Young
Ernst & Young (EY) provides audit/assurance, tax, consulting, strategy and transactions services to enterprises, financial institutions, and public‑sector clients. Structured as a global network of partner‑owned member firms, it sells professional services on a fee basis, including a dedicated Financial Services Organization for banking, insurance, and capital markets. Headquartered in London, EY was formed in 1989 from the merger of Ernst & Whinney and Arthur Young, and operates in 150+ countries.