21 hours ago
Responsibilities
- Build evaluation tooling, harnesses, and infrastructure for Siri development across Apple platforms.
- Develop systems for human evaluation, real-user feedback, reward and alignment signals, and data quality measurement.
- Design observable, reproducible, and deterministic evaluation workflows that identify quality issues early.
- Apply ML engineering techniques to model evaluation, reward modeling, prompting, RAG, and LLM-as-judge systems.
- Define success criteria, test cases, scoring methodologies, ground truth, and validation approaches based on product goals and real user workflows.
- Support deployment, monitoring, model versioning, containerization, CI/CD, and staging or provisioning environments.
- Collaborate across large, multicultural, cross-functional teams and adapt to evolving priorities and technologies.
Requirements
- Strong programming skills in one or more compiled languages, including Swift, C++, or Objective-C.
- Strong Python skills and solid computer science fundamentals in data structures, algorithms, and clean, testable code.
- Experience with backend/API development and production debugging.
- Experience working effectively within large, cross-functional organizations, with strong communication and collaboration skills.
- Bachelor’s or master’s degree in Computer Science, Machine Learning, or a related field, or equivalent experience.
- Preferred: experience evaluating ML, LLM, or agent-based systems, including metrics, scoring, trajectory and outcome analysis, prompting, RAG, or LLM-as-judge techniques.
- Preferred: understanding of reinforcement learning, transformer architectures, RLHF/RLAIF, fine-tuning, and reward modeling.
- Preferred: familiarity with PyTorch, Hugging Face, eval-driven development, statistical quality measurement, inter-rater agreement, and automated scorer validation.
- Preferred: experience with MLOps, deployment, test/evaluation environment management, containerization, CI/CD, model versioning, monitoring, cloud platforms, and repeatable staging or provisioning environments.
Tech Stack
Categories
About Apple
Apple designs and sells consumer electronics, software, and services for consumers and professionals worldwide, including iPhone, Mac, iPad, Apple Watch, and AirPods, plus platforms like iOS/macOS and services such as the App Store, iCloud, Music, and TV+. Its business combines device sales with services and subscriptions and in-house silicon design. Founded in 1976, Apple is headquartered in Cupertino, California, and trades on NASDAQ as AAPL.