QA Engineer, Voice AI Agents
Arbiter AI7 hours ago
Remote, United StatesMid Level / Senior
H1B sponsor
Base Salary
$180k - $200k/yr
Responsibilities
- Investigate live campaign issues including dropped calls, incorrect agent responses, failed handoffs, scheduling errors, and data mismatches.
- Trace failures across call recordings, transcripts, system logs, campaign configuration, and integration data.
- Write reproducible bug reports with severity, member impact, and supporting evidence.
- Triage issues and distinguish agent behavior, configuration errors, data problems, and platform bugs.
- Perform pre-launch QA of scripts, edge cases, escalation paths, workflow changes, and staff handoffs.
- Conduct ongoing conversation QA against defined standards and identify quality trends.
- Build and maintain test cases, scoring rubrics, regression suites, and release checklists.
- Automate transcript analysis, configuration validation, and outcome reconciliation where appropriate.
- Track quality metrics and report patterns to deployment, product, and engineering teams.
Requirements
- 3–5 years of experience in QA, support engineering, technical operations, or a similar role on a production software platform.
- Strong debugging skills and the ability to investigate root causes beyond the first plausible explanation.
- Experience reading logs, querying data with SQL, and working with APIs and JSON or YAML configuration.
- Experience writing trusted test plans and actionable bug reports.
- Clear written communication, precise documentation habits, and sound issue-severity judgment.
- Preferred experience includes voice AI, conversational AI, IVR, or contact center platforms.
- Preferred scripting experience includes Python or a similar language for automation and analysis.
- Preferred experience includes LLM evaluation, prompt testing, conversation quality scoring, observability tools, or call analytics tools.
- Healthcare experience, particularly payer or provider outreach, and familiarity with HIPAA and PHI handling are preferred.
Benefits
- Competitive salary and equity package.
- Paid time off.
- Health, vision, and dental insurance.
- Life and disability insurance.
- Simple IRA matching.
- Professional development budget for conferences, courses, and certifications.
- Wellness programs.
- Remote work or onsite work in the New York City or Boca Raton office.
About Arbiter AI
Arbiter is the AI-powered system that unites healthcare. By connecting patients, providers, and payers on one intelligent care orchestration platform, Arbiter ends fragmentation and ensures patients receive the right care, in the right place, at the right time.