Principal Software Engineer
Cadence Design Systems22 hours ago
Bengaluru, IndiaStaff+
Responsibilities
- Build and maintain automated validation frameworks for AI/ML models, LLM-based applications, and agent workflows.
- Develop unit, integration, end-to-end, functional, regression, performance, and safety test suites.
- Validate non-deterministic LLM outputs, hallucinations, edge cases, and multi-step agent decisions.
- Design evaluation systems using golden datasets, benchmarking pipelines, accuracy, latency, and reliability metrics.
- Integrate automated testing into CI/CD pipelines for model updates, prompt changes, and tool integrations.
- Implement observability and telemetry for traceability, monitoring, and audit readiness.
- Collaborate with ML, MLOps, Product, and Security teams to define quality gates and release criteria.
- Track quality KPIs, perform root-cause analysis, and drive continuous improvement across the AI testing lifecycle.
Requirements
- 7+ years of experience in QA, SDET, or test automation engineering.
- Strong Python programming skills and familiarity with Bash, TypeScript, or Go.
- Experience with test automation frameworks such as PyTest, Playwright, Selenium, or Cypress.
- Experience with CI/CD tools, cloud platforms, containers, LLM ecosystems, RAG architectures, vector databases, and agent frameworks.
- Experience with non-deterministic testing, LLM-as-a-judge, BLEU, ROUGE, semantic similarity scoring, prompt and agent regression testing, and AI safety testing.
- Hands-on experience with AI/ML systems or LLM-based applications, including RAG pipelines or agentic workflows.
- Experience owning AI test strategy and architecture, defining quality metrics and release gates, and delivering scalable validation pipelines for production AI systems.
- Experience supporting audit and compliance readiness.
- Bachelor's or master's degree in Computer Science, Software Engineering, or a related field.
- Preferred qualifications include enterprise or regulated-environment experience, SOC2 or ISO 27001 exposure, production observability, chaos or resilience testing, ISTQB certification, cloud/ML certifications, and AI testing certifications.
Tech Stack
AWSAzureBashCypressDockerGitHub ActionsGitLab CI/CDGoGoogle Cloud PlatformGrafanaJenkinsKubernetesPrometheuspytestPythonSeleniumTypeScript
Categories
About Cadence Design Systems
Cadence Design Systems builds electronic design automation (EDA) software, emulation/acceleration hardware, and reusable IP that help semiconductor and systems companies design, verify, and implement chips and electronic systems. It also develops computational engineering tools, including computational fluid dynamics and molecular modeling for drug discovery. Founded in 1988 and headquartered in San Jose, California, Cadence is a public company listed on NASDAQ under the ticker CDNS.