8 days ago
London, United KingdomStaff+
Responsibilities
- Define and implement end-to-end testing strategies for AI, machine learning, and generative AI solutions.
- Lead QA activities across AI product development, including quality, reliability, performance, security, and compliance.
- Design and execute test plans for LLM applications, chatbots, copilots, RAG solutions, and AI agents.
- Validate AI model accuracy, bias, fairness, explainability, robustness, and performance.
- Develop and maintain AI test automation frameworks using Python and modern testing tools.
- Perform functional, integration, regression, API, performance, and security testing for AI platforms.
- Establish test data management and synthetic data generation practices.
- Define and monitor AI quality metrics including model drift, hallucination rates, response consistency, and user-experience KPIs.
- Support CI/CD integration and quality gates for AI model deployment pipelines.
- Collaborate with data scientists, AI engineers, product owners, architects, and business stakeholders.
- Mentor QA teams and promote AI testing and automation best practices.
Requirements
- 10 years of experience in software testing, test automation, and quality engineering.
- Hands-on experience testing AI/ML and generative AI solutions.
- Strong knowledge of LLMs, RAG, prompt engineering, and AI agents.
- Experience with AI platforms such as Azure OpenAI, Microsoft Copilot Studio, OpenAI, AWS Bedrock, and Vertex AI.
- Proficiency in Python and test automation frameworks.
- Experience with API testing tools such as Postman and Rest Assured.
- Understanding of machine learning model validation techniques and evaluation metrics.
- Experience with CI/CD pipelines and Git, Azure DevOps, Jenkins, or GitHub Actions.
- Knowledge of security, privacy, ethical AI, and responsible AI principles.
- Familiarity with Azure, AWS, or GCP cloud platforms.
- Preferred experience implementing AI testing frameworks and governance controls.
- Preferred knowledge of MLOps, model lifecycle management, AI risk management, and regulatory compliance frameworks.
- Certifications in AI, cloud, quality engineering, or test automation are desirable.
Benefits
- Hybrid working across company offices, client sites, and home, with no option to work from home 100% of the time.
- Training and development opportunities including think tanks, hackathons, access to 250,000 courses, and external certifications.
- Wellbeing support including trained Mental Health Champions and Thrive and Peppy wellbeing apps.
- Inclusive employment practices through Capgemini’s Disability Confident Employer Level 2 commitment.
About Capgemini
Capgemini is a global IT services and consulting firm that delivers strategy, cloud, AI, software engineering, and managed services to large enterprises and public-sector clients. Founded in 1967 and headquartered in Paris, it is publicly traded on Euronext Paris and operates in 50+ countries. The group expanded its engineering capabilities by acquiring Altran in 2020, now operating as Capgemini Engineering.
