Senior Test Automation Engineer 100% (f/m/d)
Julius Baer22 hours ago
Madrid, SpainSenior
Responsibilities
- Define and evolve the technical test automation architecture and reusable patterns for the ML & AI ART.
- Design and automate functional, non-functional, API, UI, data pipeline, contract, regression, smoke, and release tests.
- Develop AI/ML-specific evaluation pipelines for LLM outputs, RAG systems, and agentic workflows.
- Integrate and orchestrate automated tests in GitLab CI/CD using GitLab runners, Docker, and Kubernetes.
- Plan and execute automated tests across DEV, INT, UAT, and pre-PROD environments.
- Monitor test health, investigate flaky tests, triage defects in Jira, and maintain trustworthy quality signals.
- Produce audit-ready execution evidence, reports, traceability matrices, and test progress reporting.
- Author and maintain BDD scenarios in Gherkin linked to story acceptance criteria.
- Prepare synthetic or anonymized test data and maintain datasets for AI/ML testing.
- Act as Test Automation Ambassador by advising squads on framework usage, tooling, test data, and automation design.
- Collaborate with Product Owners, Scrum Masters, ML Engineers, MLOps Engineers, Data Engineers, Architects, and Test Managers.
- Participate in PI Planning, System Demos, Inspect & Adapt, and other SAFe ceremonies.
Requirements
- Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
- Minimum 5–7 years of test automation experience with substantial hands-on Python expertise.
- Proven experience designing and owning maintainable test automation frameworks rather than only writing scripts.
- Hands-on experience integrating automated tests into CI/CD pipelines, ideally GitLab or an equivalent enterprise platform.
- Experience with Playwright-Python, Behave or pytest-bdd, requests, and pytest for UI, BDD, API, and service validation.
- Experience with distributed test execution, parallelization, flaky-test management, and modern reporting tools such as Allure or pytest-html.
- Working knowledge of Docker and familiarity with Kubernetes basics including jobs and namespaces.
- Solid understanding of Git, version control workflows, clean code principles, and code review practices.
- Understanding of API design, microservices, event-driven architectures, and authentication layers.
- Experience with Jira for story and feature tracking and test management integrations such as Xray or Agile Hive.
- Understanding of SAFe and DevOps principles; Agile Release Train experience is a plus.
- Exposure to testing AI/ML systems, including LLM output evaluation, non-deterministic responses, RAG, or agentic workflows.
- Awareness of information security, data privacy, and compliant test data handling in regulated environments.
- ISTQB Foundation certification is expected as a baseline; SAFe or DevOps certifications are a plus.
- Experience in financial services, healthcare, or pharma is strongly preferred.
- Fluency in English is required; German is a plus.
- Strong communication, collaboration, organization, ownership, and problem-solving skills.
Tech Stack
Categories
About Julius Baer
Julius Baer is a Swiss private banking group providing wealth management, investment advisory, and custody services to high- and ultra-high-net-worth clients and family offices. It earns fees and commissions on assets under management, plus trading, custody, and lending income. Founded in 1890 and headquartered in Zurich, it operates in about 25 countries across roughly 60 locations, with notable hubs in Geneva, London, Hong Kong, Singapore, and Dubai.