Machine Learning Engineer (Evals and Voice Models)
Aircall.io, Inc.6 hours ago
Base Salary
$181k - $250k/yr
Responsibilities
- Design and document evaluation frameworks for AI agents across voice, chat, and messaging.
- Train and fine-tune TTS, ASR, and speech-to-speech voice models using production and synthetic data.
- Build automated evaluation systems, benchmarks, regression suites, release gates, and repeatable pipelines.
- Design annotation workflows and calibrate LLM-as-judge systems against human raters.
- Monitor deployed AI agents for accuracy, resolution quality, safety signals, and model or data drift.
- Own metric definitions and contracts covering population, grain, rollup, and validity windows.
- Develop voice-specific evaluations for accents, languages, background noise, barge-in, DTMF, tool failures, latency, and ASR accuracy.
- Analyze model failures, system-design decisions, training pipelines, experiments, debugging processes, and optimization strategies.
Requirements
- Bachelor’s degree in Computer Science, Machine Learning, Statistics, or a related field.
- 8+ years of overall experience, including 3+ years in ML Engineering or Applied ML.
- Strong experience evaluating supervised, unsupervised, LLM, and deep learning models.
- Hands-on experience with failure analysis, LLM evaluation, and automated evaluation systems.
- Hands-on experience training or fine-tuning voice or speech models such as TTS, ASR, or speech-to-speech, including data-pipeline construction and experimentation.
- Strong communication skills for explaining complex technical concepts to technical and non-technical audiences.
- Preferred: MS or PhD in Computer Science, Machine Learning, Statistics, or a related field.
- Preferred: experience evaluating LLMs or agentic systems, including LLM-as-a-judge or RAG evaluation.
- Preferred: experience with synthetic data generation, prompt engineering, model distillation, or low-latency inference optimization for production voice agents.
Benefits
- Competitive salary package and benefits.
- Work-life balance and a fast-learning, entrepreneurial environment.
- Collaborative, multicultural workplace with 45+ nationalities.
- Opportunity to join Aircall during a major growth phase with visible impact and professional development opportunities.
Categories
About Aircall.io, Inc.
Aircall builds a cloud-based business phone and contact center platform for sales and support teams, integrating voice, SMS, WhatsApp, and social channels with 200+ CRM/helpdesk apps. It sells its software via subscription to companies that want VoIP calling, call routing/recording, analytics, and AI features like automated agents and post-call summaries. Founded in 2014 and privately held, it serves 22,000+ businesses worldwide, with operations spanning Paris (European HQ) and the Seattle area (North American HQ).