Capgemini

QET - Lead AI Test Engineer - LONDON

Capgemini
Apply
8 days ago
London, United KingdomStaff+

Responsibilities

  • Define and implement end-to-end testing strategies for AI, machine learning, and generative AI solutions.
  • Lead QA activities across AI product development, including quality, reliability, performance, security, and compliance.
  • Design and execute test plans for LLM applications, chatbots, copilots, RAG solutions, and AI agents.
  • Validate AI model accuracy, bias, fairness, explainability, robustness, and performance.
  • Develop and maintain AI test automation frameworks using Python and modern testing tools.
  • Perform functional, integration, regression, API, performance, and security testing for AI platforms.
  • Establish test data management and synthetic data generation practices.
  • Define and monitor AI quality metrics including model drift, hallucination rates, response consistency, and user-experience KPIs.
  • Support CI/CD integration and quality gates for AI model deployment pipelines.
  • Collaborate with data scientists, AI engineers, product owners, architects, and business stakeholders.
  • Mentor QA teams and promote AI testing and automation best practices.

Requirements

  • 10 years of experience in software testing, test automation, and quality engineering.
  • Hands-on experience testing AI/ML and generative AI solutions.
  • Strong knowledge of LLMs, RAG, prompt engineering, and AI agents.
  • Experience with AI platforms such as Azure OpenAI, Microsoft Copilot Studio, OpenAI, AWS Bedrock, and Vertex AI.
  • Proficiency in Python and test automation frameworks.
  • Experience with API testing tools such as Postman and Rest Assured.
  • Understanding of machine learning model validation techniques and evaluation metrics.
  • Experience with CI/CD pipelines and Git, Azure DevOps, Jenkins, or GitHub Actions.
  • Knowledge of security, privacy, ethical AI, and responsible AI principles.
  • Familiarity with Azure, AWS, or GCP cloud platforms.
  • Preferred experience implementing AI testing frameworks and governance controls.
  • Preferred knowledge of MLOps, model lifecycle management, AI risk management, and regulatory compliance frameworks.
  • Certifications in AI, cloud, quality engineering, or test automation are desirable.

Benefits

  • Hybrid working across company offices, client sites, and home, with no option to work from home 100% of the time.
  • Training and development opportunities including think tanks, hackathons, access to 250,000 courses, and external certifications.
  • Wellbeing support including trained Mental Health Champions and Thrive and Peppy wellbeing apps.
  • Inclusive employment practices through Capgemini’s Disability Confident Employer Level 2 commitment.

Tech Stack

AWSAzureGitGitHub ActionsGoogle Cloud PlatformJenkinsPostmanPython

Categories

Capgemini

About Capgemini

10,000+ employees

Capgemini is a global IT services and consulting firm that delivers strategy, cloud, AI, software engineering, and managed services to large enterprises and public-sector clients. Founded in 1967 and headquartered in Paris, it is publicly traded on Euronext Paris and operates in 50+ countries. The group expanded its engineering capabilities by acquiring Altran in 2020, now operating as Capgemini Engineering.

Contact me