Testing - AI Test Staff
Ernst and Young1 day ago
Bengaluru, IndiaStaff+
Responsibilities
- Validate Narrow AI, Generative AI, and Agentic AI models and systems for quality, reliability, and performance.
- Perform data collection, generation, augmentation, exploratory analysis, and assessment of data quality, bias, and privacy.
- Evaluate model outputs against functional, business, and performance metrics and identify defects or inconsistencies.
- Design and execute AI application test scenarios using software testing principles and AI-specific techniques.
- Apply pairwise, metamorphic, back-to-back, bias, and drift testing approaches.
- Use Python-based frameworks and automation tools to improve testing efficiency and coverage.
- Collaborate with developers, data scientists, product owners, and stakeholders to track and resolve quality issues.
- Support validation of Large Language Models and Agentic AI systems using established evaluation methodologies.
- Recommend improvements to testing processes, methodologies, and quality standards within Agile teams.
Requirements
- Proven experience testing AI models and systems.
- Strong understanding of software testing principles, methodologies, and quality engineering practices.
- Foundational knowledge of AI/ML concepts, algorithms, and model validation techniques.
- Understanding of Large Language Models, Generative AI, and Agentic AI systems.
- Experience with pairwise, metamorphic, bias, drift, and back-to-back testing.
- Proficiency in Python for test development, validation, and automation.
- Experience with Selenium, Playwright, PyTest, or similar test automation frameworks.
- Knowledge of Microsoft Azure, AWS, or Google Cloud Platform.
- Exposure to GitHub Copilot and AI-assisted development tools.
- Strong analytical, troubleshooting, problem-solving, communication, and stakeholder management skills.
- Ability to collaborate in Agile delivery environments and multidisciplinary teams.
Benefits
- Flexible work environment with professional development, mentorship, learning opportunities, and opportunities to take on leadership roles.
- Health and wellness packages, rewards, and learning benefits.
- Diverse, equitable, and inclusive workplace culture.
Tech Stack
Categories
About Ernst and Young
Ernst & Young (EY) provides audit/assurance, tax, consulting, strategy and transactions services to enterprises, financial institutions, and public‑sector clients. Structured as a global network of partner‑owned member firms, it sells professional services on a fee basis, including a dedicated Financial Services Organization for banking, insurance, and capital markets. Headquartered in London, EY was formed in 1989 from the merger of Ernst & Whinney and Arthur Young, and operates in 150+ countries.