15 hours ago
Cambridge, MA, USAMid Level
H1B sponsor
Responsibilities
- Develop metrics to evaluate safety and fairness risks in generative models and AI features.
- Design datasets, identify data needs, and scale data coverage through human and synthetic generation methods.
- Develop sampling strategies and combine human annotation with automated grading for high-quality evaluation insights.
- Implement data pipelines and collaborate cross-functionally on end-to-end safety evaluations.
- Translate evaluation findings into recommendations for product engineering teams and safety policy development.
- Develop ML-based enhancements for red teaming, model evaluation, and related processes.
- Work with highly sensitive, offensive, and controversial content.
Requirements
- A master's or PhD in computer science, linguistics, cognitive science, HCI, psychology, mathematics, physics, or a related science or technology field plus at least 4 years of relevant experience, or a bachelor's degree with at least 8 years of relevant experience.
- Experience gathering and analyzing language, image, and/or multimodal data, including LLM-generated data.
- Strong experience designing human annotation projects, writing guidelines, and handling nuanced, multi-labeled, and conflicting data.
- Proficiency in data science, machine learning, analytics, and programming with Python and Pandas.
- Strong experience with one or more plotting and visualization libraries.
- Ability to prioritize competing projects, manage milestones and schedules, and communicate with internal and external stakeholders.
- Strong skills in model quality metrics, experiment and evaluation interpretation, and executive presentation.
- Preferred qualifications include responsible AI experience, knowledge of fairness and bias in generative AI, scientific research and publication experience, and experience evaluating or developing generative models.
Tech Stack
PandasPython
Categories
About Apple
Apple designs and sells consumer electronics, software, and services for consumers and professionals worldwide, including iPhone, Mac, iPad, Apple Watch, and AirPods, plus platforms like iOS/macOS and services such as the App Store, iCloud, Music, and TV+. Its business combines device sales with services and subscriptions and in-house silicon design. Founded in 1976, Apple is headquartered in Cupertino, California, and trades on NASDAQ as AAPL.