Lead Test Framework Architect (M/F/D)
Agilent Technologies14 days ago
Barcelona, Spain or Santa Clara, CA, USAStaff+
Responsibilities
- Lead AI software development lifecycle adoption for quality engineering and drive organization-wide change through champions, enablement, and playbooks.
- Own the strategy for AI-assisted test authoring, execution, triage, maintenance, and adoption across product teams.
- Build AI-native frameworks that generate and validate unit, contract, integration, and end-to-end tests from specifications and intent.
- Integrate AI-assisted testing tools into CI/CD quality gates and build evaluation and regression frameworks for LLM- and agent-powered features.
- Define evaluation methods for non-deterministic behavior, hallucination and grounding, drift, regression, and AI observability.
- Integrate AI-assisted testing into validated environments using risk-based Computer Software Assurance, GAMP 5, the ISPE GAMP AI Guide, and IEC 62304 where applicable.
- Preserve audit trails, traceability, and data integrity for AI-generated artifacts and establish risk-scaled human review and rollback governance.
- Define platform-wide test architecture standards for Kubernetes microservices, including ephemeral environments, environment-as-code, contract testing, test data, and containerized end-to-end testing.
- Instrument and report DORA and AI-specific quality metrics, including coverage delta, defect escapes, mean time to triage, and evaluation-suite pass rates.
Requirements
- 10+ years of software engineering experience, including 3+ years in test architecture or quality engineering architecture roles.
- Daily hands-on experience with AI-assisted development and testing tools such as GitHub Copilot, Claude Code, or Cursor, including understanding of usage patterns and failure modes.
- Deep hands-on Kubernetes expertise for ephemeral namespaces, container-based test infrastructure, and environment-as-code.
- Strong experience designing multi-level automated testing frameworks for distributed systems, including unit, contract, integration, end-to-end, and performance testing.
- Experience with CI/CD pipeline architecture and quality-gate integration using GitHub Actions, Jenkins, or equivalent tools.
- Experience with Pact or consumer-driven contract testing for microservices and API-first architectures.
- Demonstrated delivery of reusable test frameworks adopted across multiple product teams, including reference implementations.
- Knowledge of testability-by-design and experience influencing software architecture to improve testability.
- Experience leading adoption and change in established engineering organizations with defensible outcome metrics.
- Preferred experience testing AI, ML, LLM, or agent-based systems using evaluation frameworks such as deepeval, promptfoo, LangSmith, Langfuse, Maxim AI, or equivalent.
- Preferred fluency in 21 CFR Part 11, GxP, GAMP 5, Computer Software Assurance versus CSV, and IEC 62304.
- Preferred experience with LLM context engineering, prompt and specification design, RAG, MCP servers, and project-memory patterns.
- Preferred experience with OpenTelemetry GenAI Semantic Conventions, performance and load testing, infrastructure-as-code, GitOps, and scientific instrument, laboratory, life sciences, or diagnostics software.
Benefits
- Agilent Result Bonus, Stock Purchase Plan, Life Insurance, Pension, Healthcare, Employee Assistance Program, holidays, and company activities.
- Commitment to work/life balance and opportunities to learn and grow.
- Full-time day-shift position with 25% travel and no end date.
- Pay and benefits vary by country, location, role level, skills, experience, and education or training.
Tech Stack
Categories