2 months ago
Bengaluru, IndiaStaff+
Responsibilities
- Create and execute test plans for single-agent, multi-agent, and orchestrated GenAI workflows.
- Validate agent instructions, tool usage, routing, handoffs, termination conditions, error handling, retries, timeouts, and fallback behavior.
- Test application code interrogation agents for security vulnerabilities, insecure coding practices, authentication and authorization issues, data exposure, resiliency gaps, and inaccurate findings.
- Test infrastructure code interrogation agents across Terraform, Kubernetes, Helm, YAML, and CI/CD configuration files.
- Validate finding normalization, deduplication, categorization, severity and priority assignment, ranking, correlation, consolidation, and avoidance of unsupported conclusions.
- Test GitLab Dedicated pipeline execution, shared CI/CD templates, triggers, reports, artifacts, logs, security gates, repository permissions, and access controls.
- Develop Python-based automated test cases, regression tests, evaluation datasets, and reusable test fixtures for prompts, tools, and agents.
Requirements
- Experience designing and executing functional, integration, regression, security, and performance tests for AI agents or complex software workflows.
- Ability to validate code-analysis and infrastructure-analysis findings for evidence, severity, confidence, business impact, remediation, and false-positive or false-negative behavior.
- Experience testing application and infrastructure configurations involving Terraform, Kubernetes, Helm, YAML, and CI/CD systems.
- Python automation experience with suitable testing frameworks and reusable test-data construction.
- Understanding of OWASP Top 10 risks, secure coding, authentication and authorization, secrets management, resiliency, logging, monitoring, and alerting.
- Experience testing GitLab Duo, AI Catalog agents, GitLab Dedicated pipelines, shared CI/CD templates, pipeline triggers, security gates, reports, artifacts, and auditability.
- Familiarity with SDLC agent use cases including requirements, UX/UI specifications, OpenAPI contracts, architecture, backend and frontend implementation, dependency curation, test generation, security review, and infrastructure planning.
Tech Stack
CSSDjangoDockerFastAPIFlaskGraphQLHelmKubernetesLessMongoDBPostgreSQLpytestPythonReactRedisReduxTerraform
Categories
About Brillio
Brillio is a privately held digital engineering and enterprise AI services firm that helps Fortune 1000 companies modernize products, data, and infrastructure. It combines consulting and managed services with an accelerator platform (ADAM) across cloud, AI, data engineering, customer experience, and infrastructure workstreams. Founded in 2014 and headquartered in Dallas, it operates delivery centers across North America, Europe, and Asia, with focus areas that include healthcare and life sciences.