1 month ago
Hyderābād, IndiaSenior
Responsibilities
- Define and own the QA strategy for AI Studio across search relevance, multi-agent coordination, schema validation, tenant isolation, and LLM output quality.
- Lead QA engineers, assign test coverage, review test plans, and enforce quality gates before releases.
- Translate architecture, component, contribution-model, and agent updates into test requirements.
- Design evaluation frameworks for non-deterministic Orchestrator outputs, including gap maps, dependency graphs, and blueprints.
- Oversee search relevance regression testing as the knowledge corpus grows.
- Define and enforce tenant-isolation testing standards across sessions, project history, and search results.
- Report solution-generation success rates, clarification rounds, agent failures, and corpus coverage gaps to engineering leadership.
Requirements
- Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field.
- 8–10 years of software quality assurance experience, including at least 3 years in a lead or principal QA role.
- Hands-on experience leading QA for an AI or data-intensive production product.
- Deep understanding of RAG testing, search relevance validation, embedding quality, hybrid search, and corpus quality measurement.
- Strong experience testing multi-agent or distributed systems across service boundaries, inter-agent contracts, and execution ordering.
- Experience building automated Python test suites and defining quality frameworks from scratch.
- Understanding of LLM evaluation methodologies and tools such as RAGAS and DeepEval.
- Experience with REST API contract testing and integration testing across multi-service architectures.
- Experience with at least one major cloud platform: AWS, Azure, or GCP.
- Strong communication and stakeholder-management skills.
- Preferred experience with agentic AI systems, LLM orchestration pipelines, OpenSearch, Elasticsearch, adversarial prompt testing, MLOps or LLMOps observability, multi-tenant SaaS security testing, Infor OS, ERP systems, enterprise workflow automation, knowledge-driven QA processes, and CI/CD or DevOps practices.
Tech Stack
Categories
About Infor
Infor is a global leader in business cloud software products for companies in industry specific markets. Infor builds complete industry suites in the cloud and efficiently deploys technology that puts the user experience first, leverages data science, and integrates easily into existing systems. Over 60,000 organizations worldwide rely on Infor to help overcome market disruptions and achieve business-wide digital transformation. Here are some key insights: • 60,000+ customers • 100+ offices • 1,700+ support experts • 2,000+ partners • 17,000+ employees • 175+ countries where customers are located • 15,000+ cloud customers • 40+ countries with Infor offices