9 hours ago
London, United KingdomSenior
Responsibilities
- Define and implement QA strategies and test plans for AI agents and LLM-powered applications.
- Design, develop, and maintain scalable automated test frameworks using languages such as Python or TypeScript.
- Collaborate with product managers, machine learning engineers, and data scientists to create test cases and validation criteria for AI features and model behaviors.
- Integrate automated quality checks into CI/CD pipelines and improve testing processes and infrastructure.
- Identify, document, prioritize, and track software defects; perform root cause analysis and debug logs and customer feedback.
- Monitor production systems and AI model performance and contribute to post-release quality validation.
- Design and maintain test data strategies and mock services for stable, isolated, and repeatable testing.
- Design, develop, or integrate agentic AI systems, AI skills, and the Model Context Protocol.
- Champion quality practices and independently drive quality initiatives across engineering teams.
Requirements
- 5+ years of hands-on experience in software quality assurance or engineering focused on complex distributed systems or AI/ML applications.
- 5+ years of experience working with GitHub Actions.
- Proficiency in Python or TypeScript for test automation and experience with modern testing frameworks such as Playwright.
- Understanding of AI/ML concepts, model evaluation metrics, data pipelines, and the challenges of testing generative AI outputs.
- Experience with AWS, Azure, or GCP and Docker or Kubernetes in a CI/CD environment is a plus.
- Strong analytical, debugging, collaboration, ownership, and independent initiative skills.
- Experience designing, developing, or integrating agentic AI systems, AI skills, and the Model Context Protocol is preferred.
Benefits
- Generous paid time off and company holidays.
- Comprehensive medical and dental insurance.
- 16 weeks of paid parental leave for all parents.
- Fertility and family planning support.
- Early-detection cancer testing through Galleri.
- Competitive pension scheme and company contribution.
- Wellness, learning and development, and annual work-life stipends.
- Company-wide and team off-sites.
- Company stock options.
- Hybrid work based out of the London, San Francisco, Seattle, or New York City hubs.
Tech Stack
Categories
About WRITER
WRITER builds an enterprise generative AI platform for creating, deploying, and supervising AI agents grounded in company data and powered by proprietary LLMs. It sells subscriptions and services to large organizations, with customers such as Accenture, Marriott, Uber, Mars, and Vanguard. Founded in 2020 and headquartered in San Francisco, the privately held company raised a Series C in 2024.