Lead AI Engineer – Quality Engineering
KMS Technology12 hours ago
Ho Chi Minh City, VietnamStaff+
Responsibilities
- Design and develop autonomous and multi-agent workflows for end-to-end test automation.
- Build AI agents for test generation, code creation, validation, debugging, failure triage, self-healing, and continuous test maintenance.
- Maintain application knowledge bases using embeddings, retrieval, and context strategies.
- Define agent instructions, guardrails, validation checkpoints, and human-in-the-loop review processes.
- Establish evaluation practices using golden datasets, regression evaluations, LLM-as-judge methods, and reliability metrics.
- Instrument and monitor agents for accuracy, reliability, latency, token cost, and maintainability in production.
- Integrate agentic solutions with UI, API, database, end-to-end testing workflows, GitHub, Jenkins, and other engineering tools.
- Set technical standards, conduct design reviews, mentor engineers, and lead the technical direction of the agentic quality engineering practice.
Requirements
- At least 6 years of software engineering experience, including experience leading technical delivery or mentoring engineers.
- At least 1.5 years of production experience building LLM-based or agentic systems beyond proof-of-concept work.
- Strong hands-on experience with LLM integration, prompt engineering, RAG, and AI-agent development.
- Proficiency in Python or TypeScript and solid experience building and consuming REST APIs.
- Hands-on experience with an agent orchestration framework such as LangGraph, OpenAI Agents SDK, CrewAI, AutoGen, or Semantic Kernel, including tool integration, structured outputs, and validation mechanisms.
- Experience designing AI evaluation datasets, measuring agent quality, and using tracing or observability tools such as LangSmith, Langfuse, or Arize Phoenix.
- Experience with vector databases, embeddings, and knowledge management for retrieval.
- Understanding of Git, CI/CD pipelines, and modern software-development practices.
- Good English communication skills for collaboration with distributed teams and stakeholders.
- Working knowledge of Playwright with TypeScript is preferred.
- Background in UI, API, database, or end-to-end test automation is preferred.
- Familiarity with Angular, React, MCP servers and clients, LLM cost and latency optimization, or Jenkins at scale is preferred.
Benefits
- Attractive salary and benefits, with an annual performance appraisal and performance bonus.
- Flexible working time and training in emerging technologies, best practices, and soft skills.
- Premium healthcare insurance for the employee and dependents.
- Short-term and long-term onsite assignment opportunities in the U.S., Canada, Europe, and Asia.
- Company trips, annual year-end party, team-building activities, and sports activities.
- Community development activities, including charity, blood donation, and public seminars.
- Free in-house entertainment facilities, gym access, coffee, and snacks.
- The role is based within KMS Technology's Vietnam operations and involves working with distributed global teams.
Tech Stack
Categories
About KMS Technology
KMS Technology provides software product development, quality assurance/testing, and AI engineering services for independent software vendors and enterprises using offshore and nearshore teams. Its consulting-led model spans DevOps, mobile app development, and automated testing, delivered through global centers in Vietnam and Mexico. Founded in 2009 and headquartered in Atlanta, the privately held company focuses on helping North American clients scale product delivery.