Base Salary
$150k - $220k/yr
Responsibilities
- Define and execute test plans across products, APIs, SDKs, model-powered features, and data platforms.
- Build and maintain automated test suites and frameworks for functional, integration, end-to-end, regression, API, browser, and service-level testing.
- Translate requirements and customer acceptance criteria into test strategies, test cases, and release gates.
- Create customer-focused and adversarial datasets, fixtures, and test environments covering edge cases, failure modes, and system limits.
- Evaluate speech-to-text, text-to-speech, and other AI behavior using metrics, expected outputs, human review, and regression coverage.
- Build test harnesses, reusable scripts, test-data tooling, result-aggregation pipelines, dashboards, and visualizations.
- Integrate automated tests, quality checks, canaries, and release validation into continuous integration and deployment workflows.
- Test data ingestion, processing, annotation, quality control, integrity, completeness, deduplication, leakage, and downstream readiness.
- Perform staging and production validation, load and reliability testing, cross-browser testing, customer-workflow testing, and user acceptance testing.
- Maintain test cases, automation coverage, documentation, and release-readiness reporting.
- Write reproducible bug reports with inputs, parameters, expected and actual results, logs or artifacts, and severity.
- Contribute to code reviews, test-design reviews, technical discussions, and engineering and QA practice improvements.
Requirements
- Bachelor's, master's, or doctoral degree in Computer Science, AI, Applied Mathematics, or a related field, or equivalent experience.
- At least 5 years of professional software or QA engineering experience, including shipping test infrastructure or evaluation systems.
- Backend or scripting experience in Python, Rust, Go, or a similar language.
- Experience designing and building automated test pipelines, evaluation frameworks, or data-processing systems.
- Strong analytical skills and comfort reasoning about metrics, thresholds, and statistical variation.
- Ability to take ownership of ambiguous technical challenges and communicate across research, engineering, and product teams.
- Preferred experience testing LLMs, RAG pipelines, agents, or multimodal models.
- Preferred experience with voice, audio, speech recognition, or real-time systems and metrics such as WER, MOS, latency, and time-to-first-byte.
- Preferred experience building test, evaluation, benchmarking, or ML infrastructure used by multiple teams or external users.
- Preferred experience with React Native, mobile applications, cloud infrastructure, containers, ephemeral test environments, monitoring tools, open-source projects, or cross-team technical bridging.
About Deepgram
Deepgram is the real-time API platform powering the trillion-dollar Voice AI economy. Backed by a $130M Series C at a $1.3B valuation, Deepgram is trusted by 200,000+ developers and 1,300+ organizations to build Voice AI products, platforms, and autonomous agents with the lowest latency, highest accuracy, and enterprise reliability. Our voice-native foundation models and runtime infrastructure have processed 50,000+ years of audio and over 1 trillion words, making Deepgram the most experienced voice AI platform in the world. Industry-leading models & platform: 👂 Nova-3 — the world’s most accurate real-time speech-to-text model 🔊 Aura-2 — professional, enterprise-grade text-to-speech 💬 Flux — the first Conversational Speech Recognition model designed to handle interruptions 🚀 Voice Agent API — enterprise-ready, real-time conversational AI 🧠 Saga — the Voice OS Beyond core infrastructure, Deepgram is expanding the Voice AI ecosystem through: 💪 Powered by Deepgram, supporting voice products built by leading AI startups and enterprise organizations 🌉 A new Voice AI Collaboration Hub in San Francisco for builders, partners, and the voice community 🍔 The acquisition of OfOne, delivering real-time Voice AI for restaurants and drive-thru operations with 95%+ containment 📃 A growing patent portfolio in Voice AI Much like APIs powered the payments and cloud economies, Deepgram is building the foundation for a trillion-dollar B2B Voice AI economy—centered on the most natural human interface: voice.