5 months ago
Bengaluru, IndiaMid Level / Senior
Responsibilities
- Design, develop, and maintain scalable, reusable test automation frameworks for Salesforce, Oracle ERP, MuleSoft, UiPath, and other enterprise applications.
- Own UI, API, backend, data validation, integration, regression, functional, exploratory, SIT, UAT, performance, and end-to-end business-flow testing.
- Create test plans, test cases, test scripts, automation suites, quality dashboards, coverage metrics, and leadership reports.
- Partner with developers, product managers, business analysts, and architects to identify testability gaps and integrate quality early in the SDLC.
- Apply AI testing methodologies including prompt testing, LLM evaluation, hallucination detection, fact-checking, model-quality testing, RAG testing, and human-in-the-loop evaluation.
- Test AI safety and robustness, including prompt-injection resistance, jailbreak detection, harmful-content filtering, PII handling, noisy inputs, and out-of-distribution inputs.
- Implement evaluator frameworks with Arize AX and use Docusign’s CrewAI-powered Quality Agent and Azure OpenAI-backed tooling for AI-assisted test generation.
- Validate datasets, data quality, source-to-target mappings, business rules, retrieval relevance, semantic search, and model behavior.
- Integrate automated tests into GitLab CI or equivalent CI/CD pipelines and conduct code reviews focused on test quality and maintainability.
- Mentor junior SDET and QA team members and communicate quality risks, defect trends, coverage gaps, and improvement recommendations.
Requirements
- Bachelor’s degree in Computer Science, Engineering, or a related field; a Master’s degree is preferred.
- 5+ years of QA/SDET experience, or 3+ years with a Master’s degree, including at least 3 years of enterprise SaaS test automation experience.
- Strong Python proficiency for automation frameworks and AI evaluation scripts.
- Hands-on experience with UiPath, Selenium, Playwright, Postman, and RestAssured.
- Hands-on Salesforce platform testing, including Apex testing across Sales, Service, and Experience Clouds.
- Experience integrating automation into CI/CD pipelines with Git-based version control.
- Strong SQL skills for data validation, source-to-target testing, and business-rule verification.
- Knowledge of prompt testing, hallucination detection, model-quality metrics, and AI safety testing.
- Understanding of QA practices, testing methodologies, and Agile/Scrum frameworks.
- Experience with Jira and Zephyr Scale for test management and traceability.
- Preferred experience with Arize AX, CrewAI, LangChain, RAG pipeline testing, Azure AI Search, MLOps, model monitoring, data drift, model bias, A/B evaluation, JMeter, k6, Gatling, or BlazeMeter.
- Salesforce certifications such as Platform Developer I or Administrator are preferred, along with strong defect triage, debugging, and root-cause analysis skills.
Benefits
- Hybrid work arrangement in Bengaluru with a minimum expectation of two days per week in the office.
- Access to Docusign’s internal AI quality tooling, including the Quality Agent, and opportunities to work on AI-powered product quality.
- Reasonable accommodations are available for qualified individuals with disabilities or religious needs during the application process.
About DocuSign
DocuSign builds e-signature, contract lifecycle management, and agreement workflow software used by businesses to prepare, sign, and manage contracts. It sells cloud-based subscriptions and APIs that integrate with systems like Salesforce, Microsoft, and Google to automate document workflows and compliance. Founded in 2003 and headquartered in San Francisco, DocuSign is a public company on NASDAQ with over 1.5 million customers in more than 180 countries.