13 hours ago
London, United KingdomSenior
Responsibilities
- Define and implement comprehensive quality assurance strategies and test plans for AI agents and LLM-powered applications.
- Design, develop, and maintain scalable automated test frameworks using Python or TypeScript and modern testing tools.
- Collaborate with product managers, machine learning engineers, data scientists, and development teams to create test cases and validation criteria for AI features and model behavior.
- Integrate automated quality checks into CI/CD pipelines and continuously improve testing processes and infrastructure.
- Identify, document, prioritize, debug, and track software defects through the full defect lifecycle.
- Monitor production systems and AI model performance and contribute to post-release quality validation.
- Design and maintain test data strategies and mock services for stable, isolated, and repeatable test execution.
- Evaluate and integrate agentic AI systems, AI skills, and the Model Context Protocol.
- Champion quality best practices and drive end-to-end quality initiatives across engineering teams.
Requirements
- 5+ years of hands-on software quality assurance or engineering experience focused on complex distributed systems or AI/ML applications.
- 5+ years of experience working with GitHub Actions.
- Proficiency in Python or TypeScript for test automation and experience with testing frameworks such as Playwright.
- Understanding of AI/ML concepts, model evaluation metrics, data pipelines, and the challenges of testing generative AI outputs.
- Experience designing, developing, or integrating agentic AI systems, AI skills, and the Model Context Protocol.
- Experience with AWS, Azure, or GCP and Docker or Kubernetes in a CI/CD environment is a plus.
- Strong analytical and debugging skills, attention to detail, independent initiative, and the ability to collaborate across teams in a fast-paced startup environment.
Benefits
- Hybrid role based out of London, San Francisco, Seattle, or New York City hubs.
- Generous paid time off and company holidays.
- Comprehensive medical and dental insurance for UK full-time employees.
- Paid parental leave for all parents for 16 weeks.
- Fertility and family planning support.
- Early-detection cancer testing through Galleri.
- Competitive pension scheme and company contribution.
- Annual wellness and learning and development stipends.
- Company-wide and team off-sites.
- Company stock options.
Tech Stack
Categories
