Base Salary
$180k - $270k/yr
Responsibilities
- Define clear, defensible, and automated metrics for voice naturalness, expressiveness, conversational cadence, and other subjective model qualities.
- Build and own distributed evaluation systems, data pipelines, and evaluation harnesses that run against live model checkpoints.
- Partner with ML researchers to establish benchmarks for Speech LLM capabilities.
- Develop dashboards and observability systems that track model health during training and surface regressions.
- Diagnose anomalous mid-training results and determine whether issues arise from model architecture, corrupted data, or infrastructure.
- Communicate statistical findings and model behavior clearly to technical and non-technical stakeholders.
- Contribute foundational evaluation infrastructure and processes to Plaud’s core SpeechLLM lab.
Requirements
- Strong software engineering skills, especially in Python, with experience building reliable distributed systems, data pipelines, or evaluation harnesses at scale.
- Ability to translate Speech LLM capabilities such as noisy-environment ASR robustness and TTS emotional steerability into measurable benchmarks.
- Experience building model-health dashboards, improving evaluation latency and signal-to-noise ratio, and identifying performance regressions.
- Ability to debug anomalous mid-training results across model architecture, data, and infrastructure causes.
- Strong communication skills for explaining statistical results and model behavior to technical and non-technical stakeholders.
- Preferred experience with traditional and modern speech metrics, including WER, CER, PESQ, and automated MOS scoring.
- Preferred experience using frontier or fine-tuned multimodal LLMs as judges for audio-model evaluation.
- Preferred experience managing large-scale crowdsourcing or preference-data collection for RLHF or DPO.
- Preferred background in statistics, experimental design, and trusted tracking dashboards such as Weights & Biases or MLflow.
- Preferred experience curating adversarial datasets covering accents, overlapping speech, and noisy acoustic environments.
Benefits
- Top-tier healthcare, dental, and vision coverage for employees and dependents with employer subsidy.
- 401(k) plan with company matching for full-time employees.
- Unlimited paid time off and 13 paid holidays.
- 12 weeks of paid new-parent leave regardless of gender.
- Hybrid schedule with a minimum of three days in the office per week in San Francisco.
- Choice of high-end laptops or workstations, annual offsites, and a fully stocked office.
- Performance bonus and equity are offered in addition to base salary.
Tech Stack
Categories
About Plaud
Plaud is building the world's most trusted AI work companion for professionals to elevate productivity and performance through note-taking solutions, loved by over 2,000,000 users worldwide since 2023. With a mission to amplify human intelligence, Plaud is building the next-generation intelligence infrastructure and interfaces to capture, extract, and utilize what you say, hear, see, and think. Plaud Inc. is a Delaware-incorporated, San Francisco-based company pushing the boundary of human–AI intelligence through a hardware–software combination. With full ISO 27001, ISO 27701, GDPR, SOC 2, HIPAA, and EN18031 compliance, Plaud is committed to the highest standards of data security and privacy protection. To learn more about Plaud, please visit https://www.plaud.ai