Head of Testing (AI)
Newcode.ai1 month ago
Dublin, Ireland +3 moreStaff+
Responsibilities
- Develop and execute the overall testing strategy for AI-powered products.
- Create testing frameworks, methodologies, test plans, test cases, documentation, and quality standards.
- Design processes and benchmarks for evaluating AI outputs, including accuracy, consistency, hallucination risks, edge cases, reliability, and performance.
- Establish repeatable regression testing, user acceptance testing, and release-validation workflows.
- Identify risks and improvement opportunities, analyze testing results, and provide actionable recommendations.
- Partner with Engineering, Product, Legal Engineering, and customer-facing teams to improve product quality and development processes.
- Build and evolve the testing function, including future team growth and operating processes.
Requirements
- 5+ years of experience in software testing, quality assurance, product quality, or a related technical role.
- Proven experience testing AI-powered products, machine learning systems, or LLM-based applications.
- Strong understanding of AI evaluation methodologies and AI system performance measurement.
- Experience designing testing frameworks and quality processes from the ground up.
- Experience working cross-functionally with Engineering, Product, and customer-facing teams.
- Familiarity with automated testing frameworks and tooling.
- Experience building quality teams or functions in a growing organization.
- Strong analytical, problem-solving, communication, and technical-translation skills, with high attention to detail.
- Nice to have: experience with legal technology, enterprise software, regulated industries, LLM applications, AI agents, generative AI workflows, or legal professionals.
Benefits
- Hybrid workplace in Dublin, Ireland.
- High-ownership role with opportunities to shape the testing function and company growth.
- Collaborative, inclusive culture focused on trust, excellence, initiative, respect, and teamwork.
- Opportunity to work with founders and peers on AI-powered legal engineering products.
Categories