7 days ago
London, United KingdomStaff+
Responsibilities
- Define and implement end-to-end testing strategies for AI, machine learning, generative AI, and intelligent automation solutions.
- Lead QA activities across AI product development, including quality, reliability, performance, security, and compliance.
- Design and execute test plans for LLM applications, chatbots, copilots, RAG solutions, and AI agents.
- Validate AI model accuracy, bias, fairness, explainability, robustness, and performance.
- Develop and maintain AI test automation frameworks using Python and modern testing tools.
- Perform functional, integration, regression, API, performance, and security testing.
- Establish test data management and synthetic data generation approaches.
- Define and monitor metrics for model drift, hallucination rates, response consistency, and user experience.
- Support CI/CD integration and quality gates for AI model deployment pipelines.
- Mentor QA teams and promote AI testing and automation best practices.
Requirements
- At least 10 years of experience in software testing, test automation, and quality engineering.
- Hands-on experience testing AI/ML and generative AI solutions.
- Strong knowledge of large language models, retrieval-augmented generation, prompt engineering, and AI agents.
- Experience with AI platforms such as Azure OpenAI, Microsoft Copilot Studio, OpenAI, AWS Bedrock, or Vertex AI.
- Proficiency in Python and test automation frameworks.
- Experience with API testing tools such as Postman or Rest Assured.
- Understanding of machine learning model validation techniques and evaluation metrics.
- Experience with CI/CD pipelines and Git, Azure DevOps, Jenkins, or GitHub Actions.
- Knowledge of security, privacy, ethical AI, and responsible AI principles.
- Familiarity with Azure, AWS, or GCP cloud platforms.
- Desirable experience includes AI testing frameworks, governance controls, MLOps, model lifecycle management, AI risk management, regulatory compliance, and relevant certifications.
Benefits
- Hybrid working across company offices, client sites, and home, with the role unable to be fully remote.
- Training and development opportunities, including think tanks, hackathons, access to 250,000 courses, and external certifications.
- Wellbeing support including Mental Health Champions and wellbeing apps such as Thrive and Peppy.
- Inclusive recruitment support through Capgemini’s Disability Confident Employer commitment.
- Opportunity to work with a global technology transformation company and support clients with AI and technology innovation.
About Capgemini
Capgemini is a global IT services and consulting firm that delivers strategy, cloud, AI, software engineering, and managed services to large enterprises and public-sector clients. Founded in 1967 and headquartered in Paris, it is publicly traded on Euronext Paris and operates in 50+ countries. The group expanded its engineering capabilities by acquiring Altran in 2020, now operating as Capgemini Engineering.