2 days ago
Porto Alegre, BrazilSenior
Responsibilities
- Contribute to QA strategy, execute test plans, and apply risk-based testing for agentic HCM features, payroll products, conversational AI, and UI surfaces.
- Design and execute tests for LLM-driven agentic flows, including multi-turn conversations, intent resolution, state transitions, edge cases, and failure modes.
- Develop, enhance, and maintain automation frameworks using Playwright and ARISE.
- Validate HCM events end to end, including conversation inputs, business logic, data persistence, GraphQL mutations, possibilities, and variant resolution.
- Integrate automated test suites with Jenkins and ECS-based CI/CD pipelines and support quality gates across DIT, FIT, STG, and PROD environments.
- Mentor QA engineers, participate in code reviews, and promote test automation best practices in a TypeScript-first environment.
- Collaborate with product managers, developers, and release managers on quality criteria, AI behavior, risk areas, and release readiness.
- Define and track QA KPIs such as pass rates, automation coverage, agentic flow accuracy, defect escape rate, and mean time to detect.
- Evaluate QA tooling and apply techniques such as prompt regression testing, conversation replay, and LLM output evaluation.
Requirements
- At least 5 years of QA engineering experience with substantial hands-on test automation and modern software development practices.
- Hands-on experience testing LLM-powered conversational or agentic applications, including state machines, intent classification, multi-turn context, and non-deterministic outputs.
- Experience with the ARISE test automation platform, including authoring, tagging, and executing test suites across environments.
- Strong experience building Playwright test suites, preferably with TypeScript, including auto-waiting, network interception, API context, and trace viewer.
- Experience testing GraphQL APIs, including mutations, queries, schema validation, and data integrity checks.
- Experience integrating test frameworks with Jenkins or an equivalent system, including artifact management and test reporting pipelines.
- Strong TypeScript or JavaScript coding skills for maintainable, reviewable test automation code.
- Proficiency with REST and GraphQL API testing and event-driven microservice architectures.
- Familiarity with AWS services and multi-environment test management.
- A bachelor's degree in a related discipline is desired.
- Analytical, communication, ownership, and collaboration skills, including the ability to define pass/fail criteria for probabilistic LLM outputs.
- Experience with Agile/SAFe delivery teams, accessibility testing using WCAG 2.1, and internationalization testing is desirable.
Benefits
- Hybrid work model in the Porto Alegre office, with up to two days of home office per week; candidates must live nearby.
- Competitive compensation, collaborative engineering culture, career growth opportunities, and the opportunity to work on AI-powered agentic HCM systems at enterprise scale.
- The role may involve periodic background checks due to access to confidential and sensitive customer and employee information.
Tech Stack
Categories
About ADP
ADP builds human capital management software and outsourced services for businesses of all sizes, including payroll, tax filing, time, benefits, and talent tools. Its revenue comes from subscriptions and service fees, and it also offers a professional employer organization (PEO) and compliance support. Founded in 1949 and headquartered in Roseland, New Jersey, ADP is a public company traded on NASDAQ under the ticker ADP.