Manager | Automation Testing | Mumbai | Engineering | Quality Engineering
Deloitte Advisory1 month ago
Mumbai, IndiaStaff+
Responsibilities
- Own the end-to-end test strategy, automation roadmap, quality gates, and release-readiness criteria.
- Design and maintain scalable automation frameworks using Playwright with Python, Pytest, and Behave.
- Automate UI, API, integration, regression, end-to-end, and UAT testing, including maintaining Selenium regression suites.
- Validate AI and LLM systems for accuracy, relevance, hallucination, bias, safety, context retention, and fallback handling.
- Test prompt injection, harmful prompts, edge cases, RAG workflows, and AI agent integrations.
- Build LLM evaluation frameworks, benchmark datasets, scorecards, and regression suites.
- Integrate automated tests into CI/CD pipelines and implement continuous quality gates.
- Conduct basic performance testing covering response-time percentiles, throughput, load, and SLA compliance.
- Debug issues using logs, traces, database validation, and backend monitoring.
- Define quality dashboards and ensure traceability, auditability, and compliance.
- Mentor QA engineers and collaborate with Product, Engineering, Data Science, Security, DevOps, and business teams.
Requirements
- At least 9 years of experience in Quality Engineering and Test Automation.
- Strong hands-on expertise in Playwright with Python.
- Experience building scalable automation frameworks using Python, Pytest, and Behave.
- Experience testing AI, generative AI, or LLM-powered applications.
- Strong API testing experience using Playwright, Postman, Requests, or REST-assured equivalents.
- Experience with BDD, Git, CI/CD, and Agile delivery practices.
- Knowledge of AI evaluation, prompt testing, RAG validation, hallucination testing, and Responsible AI.
- Strong problem-solving, mentoring, communication, and stakeholder-management skills.
- Bachelor's degree in computer science, information technology, or a related field.
About Deloitte Advisory
Deloitte Advisory provides consulting and managed services that help organizations manage risk, strengthen controls, investigate fraud, meet regulatory requirements, and execute transactions using analytics and industry expertise. Offerings span cybersecurity, M&A due diligence, finance and regulatory transformation, internal audit, and tech-enabled platforms. It is part of Deloitte’s global network (founded in 1845 and privately held), with U.S. member-firm teams serving corporations and government agencies across sectors.