
RC-REGULATORY COMPLIANCE-GxP AI Model Consultant-Senior
Ernst and Young16 days ago
Hyderābād, IndiaSenior
Responsibilities
- Develop AI/ML and agentic AI strategies through analysis, validation insights, and readiness assessments.
- Integrate, evaluate, and validate AI/ML models throughout development, training, testing, deployment, and post-deployment monitoring.
- Design and execute testing strategies for AI agents and multi-agent workflows, including orchestration, tool invocation, output validation, and edge cases.
- Create or review validation plans, requirements, risk assessments, traceability matrices, test protocols, test evidence, deviation records, and validation summary reports.
- Define acceptance criteria for data quality and model performance using measures such as accuracy, sensitivity, specificity, precision, recall, F1 score, calibration, and error rates.
- Perform or independently review functional, integration, regression, negative, boundary, security-role, audit-trail, data-reconciliation, and user-acceptance testing.
- Evaluate reliability, robustness, bias, subgroup performance, explainability, stress behavior, reproducibility, and residual risk.
- Define monitoring for model performance, data and concept drift, anomalous outputs, overrides, human review, complaints, and incidents.
- Assess model updates, retraining, prompt and knowledge-base changes, infrastructure changes, and vendor releases through change control and impact assessment.
- Perform root-cause analysis for hallucinations, inaccurate outputs, agent failures, latency issues, and unexpected model behavior.
- Implement AI observability capabilities for tracing, evaluation logging, dashboards, alerting, failure analysis, and audit-ready evidence.
- Collaborate with data scientists, engineers, clients, and stakeholders to identify model improvements, risk mitigations, and business solutions.
Requirements
- Bachelor’s or master’s degree in life sciences, engineering, or a related field.
- At least 5 years of experience in AI/ML, GenAI, software testing, validation, or quality engineering roles.
- Strong understanding of ML models, LLMs, GenAI workflows, agentic systems, and model evaluation techniques.
- Hands-on experience with precision, recall, confusion matrices, ROC/AUC, supervised learning, and unsupervised learning models.
- Practical experience with LangChain, LangGraph, and Langfuse, including agent workflows, observability, traceability, and evaluation logging.
- Experience evaluating GenAI and LLM-based applications, agents, RAG solutions, retrieval quality, citation accuracy, and response grounding.
- Experience creating user stories, test scripts, validation plans, test cases, acceptance criteria, AI prototypes, test harnesses, and demo environments.
- Ability to identify bias, drift, instability, performance degradation, and reliability issues across datasets and failure scenarios.
- Exposure to Azure, AWS, or GCP cloud ecosystems.
- Familiarity with data integrity principles and electronic records/e-signatures compliance.
- Consulting, client-facing, stakeholder management, documentation, troubleshooting, communication, and presentation skills.
- Familiarity with version control, prompt engineering fundamentals, and AI-assisted testing tools is preferred.
- Business development, client engagement, thought leadership, process optimization, market awareness, and technology transformation experience are desirable.
Benefits
- Support, coaching, and feedback from experienced colleagues.
- Opportunities to develop new skills and progress your career.
- Individual progression planning, education, coaching, and practical learning experiences.
- Flexible autonomy to handle the role in a way that is right for you.
- Opportunities to work with global EY Consulting practices and multidisciplinary teams.
Tech Stack
Categories
About Ernst and Young
Ernst & Young (EY) provides audit/assurance, tax, consulting, strategy and transactions services to enterprises, financial institutions, and public‑sector clients. Structured as a global network of partner‑owned member firms, it sells professional services on a fee basis, including a dedicated Financial Services Organization for banking, insurance, and capital markets. Headquartered in London, EY was formed in 1989 from the merger of Ernst & Whinney and Arthur Young, and operates in 150+ countries.