13 hours ago
Responsibilities
- Define evaluation criteria, metrics frameworks, benchmarks, and quality standards for production ML models.
- Design adversarial test strategies, aggressor scenarios, edge-case corpora, and behavioral testing approaches to expose model failure modes.
- Evaluate model accuracy, precision-recall tradeoffs, calibration, fairness, robustness, distribution shift, out-of-distribution generalization, and temporal drift.
- Own model quality sign-off and make final readiness decisions before models ship to users.
- Translate metric results into product-quality narratives and recommendations for engineering and executive audiences.
- Collaborate with ML Engineering, Product, Privacy, and Legal teams across the model launch process.
Requirements
- At least five years of hands-on machine learning experience with deep expertise in model evaluation, offline metric design, and behavioral testing.
- A master's degree in Machine Learning, Computer Science, Statistics, Applied Mathematics, or a related technical field is strongly preferred; a bachelor's degree plus seven or more years of relevant hands-on experience may be considered.
- Strong programming skills in Python and fluency with evaluation tooling, data pipelines, and experiment tracking such as MLflow or W&B.
- Proven experience designing evaluation frameworks for production ML systems beyond basic accuracy and F1 metrics.
- Experience testing distribution shift, out-of-distribution generalization, and temporal drift in deployed models.
- Experience constructing adversarial test suites and edge-case corpora that surface model failure modes.
- Strong communication skills and the ability to connect model metrics to product and user-trust outcomes.
- Experience owning model quality sign-off in a cross-functional launch process.
- Preferred experience includes structured or semi-structured document understanding, OCR pipelines, financial data extraction, Bayesian or causal graph-based data generation, causal fairness evaluation, privacy-constrained or on-device inference, confidence calibration, uncertainty quantification, or financial services and payment products.
- A PhD in Computer Science, Data Science, Statistics, AI/ML, or a related field is preferred.
Tech Stack
MLflowPython
Categories
About Apple
Apple designs and sells consumer electronics, software, and services for consumers and professionals worldwide, including iPhone, Mac, iPad, Apple Watch, and AirPods, plus platforms like iOS/macOS and services such as the App Store, iCloud, Music, and TV+. Its business combines device sales with services and subscriptions and in-house silicon design. Founded in 1976, Apple is headquartered in Cupertino, California, and trades on NASDAQ as AAPL.