2 days ago
Vancouver, Canada
Senior
H1B Sponsor
Responsibilities
- Own end-to-end quality for agentic features and workflows.
- Design and build automation tooling and frameworks for AI systems.
- Develop and maintain evaluation frameworks to measure response quality.
- Drive automation coverage for critical AI workflows.
- Integrate AI quality checks into CI/CD pipelines.
- Build tooling for LLM observability and debugging.
- Partner with Applied AI teams on prompt engineering and evaluation strategies.
- Design and execute performance and load tests for AI services.
- Identify and mitigate risks related to AI systems.
- Define and track AI quality KPIs.
- Participate in design and architecture reviews.
- Mentor engineers and contribute to AI quality engineering practices.
Requirements
- 5+ years of experience in software engineering or SDET roles.
- Strong programming skills in Python, Java, or JavaScript.
- Experience testing distributed, cloud-native SaaS systems and APIs.
- Proficiency in coding with AI agents.
- Hands-on exposure to LLMs or AI/ML systems.
- Understanding of non-deterministic systems and probabilistic testing.
- Experience building test frameworks and scalable automation systems.
- Familiarity with AI evaluation techniques.
- Experience with CI/CD pipelines.
- Strong collaboration skills across distributed teams.
- Bachelor’s degree in Computer Science or equivalent experience.
Benefits
- Competitive benefits and perks.
- Robust training program for professional development.
- Inclusive office environment designed for collaboration.
- Recognized as a Great Place to Work.
Tech Stack
ElasticsearchGoGoogle Cloud PlatformJavaJavaScriptKubernetesPythonReactRedisVue.js
Categories
AI & MLBackendFrontendTesting