5 hours ago
Remote, IndiaSenior
Responsibilities
- Design and maintain automated frontend, backend, contract, workflow, multi-tenant isolation, sandbox, and agent behavior test suites.
- Test Temporal workflows through worker failures, provider outages, retry storms, idempotency checks, and compensation or saga verification.
- Develop statistical, semantic, adversarial, and human-in-the-loop testing strategies for LLM-driven behavior.
- Build synthetic-data pipelines, evaluation infrastructure, mocked provider layers, and CI integrations for repeatable quality measurement.
- Automate performance, concurrency, backpressure, failover, tracing, token-accounting, and alerting verification against platform NFRs.
- Integrate tests into CI/CD as release gates and produce auditable evidence for client milestone sign-offs.
- Maintain test environments and reusable frameworks while mentoring mid-level QA engineers and contributing to internal AI testing practices.
Requirements
- At least 8 years of test automation experience, including API testing and development of reusable test frameworks.
- Hands-on experience testing LLM-powered or other non-deterministic systems using statistical assertions, semantic scoring, and model-variance management.
- Experience testing asynchronous or event-driven systems, failure injection, idempotency, and eventual consistency; Temporal or a similar workflow engine is preferred.
- Strong CI/CD integration experience, including GitHub Actions or an equivalent, and release-gate design.
- Solid SQL and data-validation skills, including synthetic test-data pipelines and dataset-quality measurement.
- Security-testing or adversarial testing experience, including prompt-injection techniques or an AppSec background.
- Strong judgment and communication skills for defending quality evidence to non-engineering stakeholders.
- Mandatory proficiency in Playwright and TypeScript for automated test-suite development.
- Deep understanding of LLM behavior, prompt sensitivity, and common RAG failure modes.
- Strong statistical-analysis skills for interpreting evaluation scores and giving actionable feedback to AI engineers.
- Preferred experience with k6 or Locust, multi-tenant SaaS isolation, evaluation and observability stacks, compliance or accessibility testing, document-extraction systems, LLM-generated test assets, high-concurrency background processing, or qTest.
- Commercial real estate, lease accounting, CAM reconciliation, OCR, or layout-parsing experience is desirable.
Benefits
- Friendly, open-door community and collaboration with experienced professionals.
- Opportunities to work on large-scale, globally impactful projects and explore different domains through internal mobility.
- Internal meetups, conferences, workshops, Udemy access, language courses, and company-paid certifications.
- Company-paid medical insurance, mental health support, and financial and legal consultations.
- Full-time position based in India.
Tech Stack
Categories
About Ciklum
Ciklum is an IT services firm that builds custom digital products and provides nearshore engineering teams, QA, DevOps, data analytics, and UX for enterprises and high-growth startups. It operates a services model spanning product engineering, e-commerce solutions, and managed delivery centers. Founded in 2002 and headquartered in London, the privately held company serves global clients including Just Eat and Zurich Insurance.