5 months ago
Remote, EMEA +2 moreSenior
Responsibilities
- Design model evaluation pipelines for development and production environments.
- Design user studies for subjective model evaluations.
- Convert requirements into measurable metrics.
- Develop automated dashboards to visualize model performance and compare results.
- Train new models to capture new evaluation metrics.
- Collaborate with model, data, and product teams to improve models and measure product requirements.
- Help grow the evaluation team as its founding member and lead it in the future.
Requirements
- Strong experience and intuition designing metrics that capture model performance.
- Strong experience designing user studies on Mechanical Turk or similar platforms.
- Strong experience training and fine-tuning models for evaluation.
- Strong statistical knowledge and experience comparing evaluation results statistically to make decisions.
- Very strong engineering and programming skills.
- Experience training ASR and TTS models.
- Experience working on large-scale machine-learning problems, including models larger than 3B and datasets exceeding 1 million hours of data.
Categories
About Cantina
Cantina Labs is a social AI company, developing a suite of advanced real-time models that push the boundaries of expression, personality, and realism. We bring characters to life, transforming how people tell stories, connect, and create. We build and power ecosystems. Cantina, our flagship social AI platform, is just the beginning.