Product Validation Engineer II: AI Agent Quality Assurance (QA)
Cadence Design Systems2 days ago
Belo Horizonte, BrazilMid Level
Responsibilities
- Validate AI agent products and customer workflows through structured test planning and execution.
- Test agent behavior across prompts, tool usage, retrieval quality, workflow execution, error handling, and response consistency.
- Evaluate outputs using quality rubrics, benchmarks, regression checks, and metrics including accuracy, relevance, safety, latency, and task success.
- Create testcase suites and evaluation datasets covering customer scenarios, edge cases, and domain-specific workflows.
- Document defects, analyze failures, support root-cause triage, and verify fixes through repeatable regression testing.
- Collaborate with product and engineering teams to improve AI agent quality, release readiness, and customer experience.
Requirements
- Bachelor’s degree in Computer Science, Computer Engineering, Electrical Engineering, or equivalent experience.
- At least 2 years of experience in product validation, software QA, test engineering, software development, or engineering productivity tools.
- Hands-on experience writing test plans and testcases, executing validation cycles, reporting defects, and verifying fixes.
- Familiarity with AI/LLM concepts, AI agents, prompt behavior, RAG workflows, and common AI system failure modes.
- Ability to evaluate AI agent outputs using clear quality criteria and communicate findings to technical and product stakeholders.
- Experience with Python, scripting, or automation for testing, data preparation, log analysis, or reporting.
- Good spoken and written English and Portuguese, with strong attention to detail and a collaborative mindset.
- Preferred experience includes validating AI agents, copilots, conversational products, or workflow automation tools.
- Preferred knowledge includes AI evaluation frameworks, benchmarking, red-team testing, evaluation datasets, vector databases, embeddings, agent orchestration frameworks, EDA, VLSI design, design verification, emulation, Palladium, or Protium workflows.
Benefits
- Competitive benefits.
- CLT employment category and a 40-hour-per-week employment term.
- Work location: Av Contorno 5800, Belo Horizonte, Minas Gerais, Brazil.
About Cadence Design Systems
Cadence Design Systems builds electronic design automation (EDA) software, emulation/acceleration hardware, and reusable IP that help semiconductor and systems companies design, verify, and implement chips and electronic systems. It also develops computational engineering tools, including computational fluid dynamics and molecular modeling for drug discovery. Founded in 1988 and headquartered in San Jose, California, Cadence is a public company listed on NASDAQ under the ticker CDNS.