16 hours ago
Responsibilities
- Build and curate evaluation datasets that reflect real-world speech usage, including personalized named-entity queries and multi-turn conversations.
- Design metrics and automated judges for evaluating speech LLM accuracy, robustness, and conversational quality.
- Develop data and evaluation pipelines for large-scale audio and speech model assessment.
- Partner with modeling, infrastructure, and product teams to ensure models are evaluated quickly, consistently, and rigorously.
- Communicate evaluation results and actionable insights to technical and non-technical audiences.
Requirements
- Bachelor’s degree in Computer Science, Electrical Engineering, or a related field, or equivalent practical experience.
- Experience building or working with text, speech, or audio evaluation pipelines and metrics.
- Proficiency in Python and experience building data processing pipelines at scale.
- Experience curating or annotating datasets for machine learning evaluation or training.
- Working knowledge of statistics for measuring model performance and interpreting evaluation results.
- Familiarity with large language model evaluation techniques, including automated LLM-as-judge and human evaluation methods.
- Strong written and verbal communication skills.
- Preferred qualifications include audio-native or multimodal LLM evaluation, large-scale human evaluation studies, speech personalization and named-entity evaluation, multilingual audio dataset development, distributed data processing with frameworks such as Spark, and contributions to speech, audio ML, or NLP evaluation.
Tech Stack
Categories
About Apple
Apple designs and sells consumer electronics, software, and services for consumers and professionals worldwide, including iPhone, Mac, iPad, Apple Watch, and AirPods, plus platforms like iOS/macOS and services such as the App Store, iCloud, Music, and TV+. Its business combines device sales with services and subscriptions and in-house silicon design. Founded in 1976, Apple is headquartered in Cupertino, California, and trades on NASDAQ as AAPL.