Ernst and Young

RC-REGULATORY COMPLIANCE-GxP AI Model Consultant-Senior

Ernst and Young
Apply
16 days ago
Hyderābād, IndiaSenior

Responsibilities

  • Develop AI/ML and agentic AI strategies through analysis, validation insights, and readiness assessments.
  • Integrate, evaluate, and validate AI/ML models throughout development, training, testing, deployment, and post-deployment monitoring.
  • Design and execute testing strategies for AI agents and multi-agent workflows, including orchestration, tool invocation, output validation, and edge cases.
  • Create or review validation plans, requirements, risk assessments, traceability matrices, test protocols, test evidence, deviation records, and validation summary reports.
  • Define acceptance criteria for data quality and model performance using measures such as accuracy, sensitivity, specificity, precision, recall, F1 score, calibration, and error rates.
  • Perform or independently review functional, integration, regression, negative, boundary, security-role, audit-trail, data-reconciliation, and user-acceptance testing.
  • Evaluate reliability, robustness, bias, subgroup performance, explainability, stress behavior, reproducibility, and residual risk.
  • Define monitoring for model performance, data and concept drift, anomalous outputs, overrides, human review, complaints, and incidents.
  • Assess model updates, retraining, prompt and knowledge-base changes, infrastructure changes, and vendor releases through change control and impact assessment.
  • Perform root-cause analysis for hallucinations, inaccurate outputs, agent failures, latency issues, and unexpected model behavior.
  • Implement AI observability capabilities for tracing, evaluation logging, dashboards, alerting, failure analysis, and audit-ready evidence.
  • Collaborate with data scientists, engineers, clients, and stakeholders to identify model improvements, risk mitigations, and business solutions.

Requirements

  • Bachelor’s or master’s degree in life sciences, engineering, or a related field.
  • At least 5 years of experience in AI/ML, GenAI, software testing, validation, or quality engineering roles.
  • Strong understanding of ML models, LLMs, GenAI workflows, agentic systems, and model evaluation techniques.
  • Hands-on experience with precision, recall, confusion matrices, ROC/AUC, supervised learning, and unsupervised learning models.
  • Practical experience with LangChain, LangGraph, and Langfuse, including agent workflows, observability, traceability, and evaluation logging.
  • Experience evaluating GenAI and LLM-based applications, agents, RAG solutions, retrieval quality, citation accuracy, and response grounding.
  • Experience creating user stories, test scripts, validation plans, test cases, acceptance criteria, AI prototypes, test harnesses, and demo environments.
  • Ability to identify bias, drift, instability, performance degradation, and reliability issues across datasets and failure scenarios.
  • Exposure to Azure, AWS, or GCP cloud ecosystems.
  • Familiarity with data integrity principles and electronic records/e-signatures compliance.
  • Consulting, client-facing, stakeholder management, documentation, troubleshooting, communication, and presentation skills.
  • Familiarity with version control, prompt engineering fundamentals, and AI-assisted testing tools is preferred.
  • Business development, client engagement, thought leadership, process optimization, market awareness, and technology transformation experience are desirable.

Benefits

  • Support, coaching, and feedback from experienced colleagues.
  • Opportunities to develop new skills and progress your career.
  • Individual progression planning, education, coaching, and practical learning experiences.
  • Flexible autonomy to handle the role in a way that is right for you.
  • Opportunities to work with global EY Consulting practices and multidisciplinary teams.

Categories

Ernst and Young

About Ernst and Young

10,000+ employees

Ernst & Young (EY) provides audit/assurance, tax, consulting, strategy and transactions services to enterprises, financial institutions, and public‑sector clients. Structured as a global network of partner‑owned member firms, it sells professional services on a fee basis, including a dedicated Financial Services Organization for banking, insurance, and capital markets. Headquartered in London, EY was formed in 1989 from the merger of Ernst & Whinney and Arthur Young, and operates in 150+ countries.

Contact me