Machine Learning Quality Engineer
London Stock Exchange Group2 months ago
Hyderābād, India or London, United KingdomSenior
Responsibilities
- Define and implement evaluation frameworks for MCP tools and Skills.
- Build and maintain automated test pipelines for agentic behaviors, including tool invocation and multi-step workflows.
- Evaluate and mitigate hallucinations, tool misuse, invalid inputs, latency amplification, and other agentic failure modes.
- Produce testing evidence for Permit to Build and Permit to Operate processes.
- Partner with ML Engineers to embed testability and evaluation hooks into MCP servers and Skills.
- Help define the long-term quality and governance model for federated MCP contributions.
Requirements
- Strong Python experience for test harnesses and automation.
- Experience with LLM and RAG evaluation frameworks or custom evaluation pipelines.
- Test automation expertise across unit, integration, and regression testing.
- Understanding of agentic system risks and failure modes.
- Ability to assess solutions against governance, security, and audit expectations.
- Experience working in regulated or highly governed engineering environments.
Benefits
- Healthcare benefits, retirement planning, paid volunteering days, and wellbeing initiatives.
- Collaborative and creative culture with opportunities to contribute ideas.
- Equal-opportunity employer with reasonable accommodations for religious practices, mental health needs, and disabilities.
- Opportunity to work in a global organization of approximately 25,000 people across 65 countries.