11 hours ago
Hyderābād, IndiaSenior
Responsibilities
- Design testing strategies for LLM-based applications, RAG pipelines, and autonomous multi-agent systems.
- Build and maintain full-stack automated tests across Python services and TypeScript/React/Angular frontends.
- Develop AI evaluation frameworks, test documentation, test cases, and testing guidelines.
- Test edge cases, model bias, fairness, output consistency, contextual accuracy, prompt injection, and data leakage.
- Conduct performance, load, response-time, and resource-utilization testing for generative AI applications.
- Integrate automated quality suites into GitHub Actions and CI/CD pipelines.
- Curate and version golden datasets and monitor production behavior, drift, and user feedback through observability tools.
- Work with Agile teams in regulated environments using GxP, CSV, and ITSM processes.
Requirements
- Bachelor's or master's degree in Computer Science, Software Engineering, or an equivalent field.
- At least 7 years of software testing experience, including at least 1 year focused on generative AI or agentic applications.
- Advanced Python proficiency and experience with pytest, automated test orchestration, dataset processing, and AI-service integration.
- Strong TypeScript test automation experience with tools such as Playwright, Jest, and Cypress.
- Experience implementing AI evaluation harnesses such as RAGAS, DeepEval, and Promptfoo.
- Ability to validate autonomous agent loops, tool-use accuracy, reasoning steps, and goal-directed performance.
- Experience with golden datasets, LLM architectures, RAG systems, AI failure modes, and AI application testing.
- Experience integrating test suites with GitHub Actions or other CI/CD pipelines.
- Preferred experience with AWS, Grafana, Jira, GitHub, LangSmith, and Arize Phoenix.
- Understanding of security testing, regulated environments, AI quality standards, and compliance practices.
- Strong analytical, communication, documentation, and problem-solving skills; English proficiency at C1+ level.
Benefits
- Hyderabad, India location with working hours structured for Central European Time overlap, typically extending into the IST evening.
- Opportunity to work on applied AI and agentic systems in a regulated healthcare environment.
- Roche provides an inclusive, equal-opportunity workplace that values personal expression and diverse perspectives.
Tech Stack
Categories
About Roche
Roche is a Swiss healthcare company that develops and sells prescription pharmaceuticals and in vitro diagnostic systems for hospitals, laboratories, and clinicians. Its business spans two core divisions—Pharmaceuticals and Diagnostics—with strengths in oncology, immunology, neuroscience, infectious diseases, and diabetes care. Founded in 1896 and headquartered in Basel, Switzerland, Roche also advances personalized healthcare by pairing therapies with companion diagnostics.