4 months ago
Bengaluru, IndiaStaff+
Responsibilities
- Build automated testing suites for hallucinations, bias, toxicity, prompt injection, RAG quality, multi-agent workflows, conversational journeys, and prompt regressions.
- Validate AI data outputs, ingestion and transformation pipelines, feature stores, vector database indexing, embeddings, synthetic datasets, and privacy compliance.
- Track ML and deep learning metrics, detect data and concept drift, and validate live inference endpoints.
- Build scalable automation frameworks for APIs, backend services, and model endpoints.
- Integrate AI evaluation and data quality suites into MLOps and CI/CD pipelines so quality failures block releases.
- Define AI quality KPIs and communicate release readiness to engineering and product teams.
Requirements
- 8+ years of SDET experience.
- Expert-level Python experience for test automation, evaluation pipelines, and data analysis.
- SQL experience for data validation, ground-truth queries, and pipeline quality checks.
- Experience with LLM evaluation, agent workflow testing, prompt tracing, API automation, model monitoring, and CI/CD automation.
- Preferred experience with AWS Bedrock, Azure OpenAI, GCP Vertex AI, Kubeflow, Weights & Biases, Feast, Scikit-learn, TensorFlow, PyTorch, Kubernetes, Terraform, Playwright, Cypress, Locust, or JMeter.
- Familiarity with statistical hypothesis testing and synthetic data generation for AI evaluation.
Tech Stack
Categories
About Tekion
Tekion builds an AI-native, cloud platform for automotive retail that unifies dealers, OEMs, and partners. Its products include Automotive Retail Cloud (a dealership management system for retailers), Automotive Enterprise Cloud for manufacturers, and Automotive Partner Cloud for integrations, delivered as subscription software. Privately held and headquartered in Pleasanton, California, Tekion raised private equity funding in 2024.