EY - GDS Consulting - AIA - Gen AI - Senior
Ernst and Young14 days ago
Mumbai, IndiaSenior
Responsibilities
- Design, develop, and execute testing strategies for AI, GenAI, Agentic AI, Agentic RAG, conversational AI, multimodal AI, and enterprise AI systems.
- Build automated Python test frameworks and validation utilities for AI workflows, APIs, models, orchestration pipelines, and enterprise applications.
- Evaluate LLM outputs, retrieval pipelines, vector search, RAG applications, agent workflows, prompt engineering, reasoning, hallucinations, bias, toxicity, safety, and compliance.
- Perform functional, integration, regression, performance, scalability, reliability, security, adversarial, prompt injection, jailbreak, and AI penetration testing.
- Use Ragas and similar evaluation frameworks to measure retrieval quality, answer correctness, faithfulness, relevance, grounding, and response effectiveness.
- Validate AI integrations with APIs, enterprise applications, databases, cloud platforms, and Azure AI services.
- Develop quality metrics, dashboards, defect reporting processes, reusable testing assets, governance controls, and continuous quality improvement practices.
- Collaborate with AI engineers, architects, data scientists, product teams, and business stakeholders on design reviews, risk assessments, model evaluations, and production-readiness validation.
- Support AI testing presales, innovation initiatives, technical demonstrations, client workshops, and mentorship of junior QA engineers.
Requirements
- BTech, bachelor’s, master’s, or PhD in computer science, data science, engineering, or a related field.
- 5–9+ years of total professional experience, including at least 3 years of directly relevant hands-on experience in AI/ML, applied AI, or GenAI systems.
- Professional implementation experience with production, pre-production, or enterprise pilot AI applications; personal projects, cloned demos, certifications, and tutorials alone are insufficient.
- Strong hands-on experience in at least three AI areas, with meaningful ownership in at least two, including LLM applications, retrieval pipelines, agent orchestration, evaluation or guardrail systems, model adaptation, cloud deployment, conversational AI, multimodal AI, or backend engineering.
- Strong software engineering foundations and experience building reliable, maintainable AI services and automated test suites using Python.
- Experience with Ragas or similar evaluation frameworks, Agentic RAG, retrieval augmentation, vector databases, embeddings, enterprise search, AI observability, monitoring, and performance benchmarking.
- Experience with Microsoft Azure AI Platform, Azure AI services, FastAPI-based services, API testing frameworks, and automated API testing strategies.
- Knowledge of Responsible AI, AI safety, governance, regulatory compliance, secure development, CI/CD testing integration, and quality engineering practices.
- Strong analytical, troubleshooting, root-cause analysis, communication, presentation, and business problem-solving skills.
Benefits
- Work on AI-driven innovation and projects for global clients across multiple sectors.
- Collaborate with AI experts, analytics leaders, and industry specialists in a global delivery network spanning Argentina, China, India, the Philippines, Poland, and the UK.
- Access continuous learning, coaching, leadership development, and flexible tools to support career growth.
- Work in a diverse and inclusive culture with opportunities to collaborate across EY service lines, geographies, and sectors.
About Ernst and Young
Ernst & Young (EY) provides audit/assurance, tax, consulting, strategy and transactions services to enterprises, financial institutions, and public‑sector clients. Structured as a global network of partner‑owned member firms, it sells professional services on a fee basis, including a dedicated Financial Services Organization for banking, insurance, and capital markets. Headquartered in London, EY was formed in 1989 from the merger of Ernst & Whinney and Arthur Young, and operates in 150+ countries.