3 days ago
Vancouver, CanadaSenior
Responsibilities
- Define evaluation strategies and develop evaluation scripts and automation for machine learning models.
- Lead testing activities, including workflow development, regression testing, bug filing, triage, and driving issues to resolution.
- Prepare and collect evaluation data, analyze annotations and test results, and ensure data quality.
- Develop tools and workflows using LLMs and generative AI frameworks to automate model evaluation and improve test efficiency and accuracy.
- Work with annotation operations and cross-functional team members to define requirements and appropriate testing processes.
- Communicate project progress, risks, and technical findings through regular updates and presentations.
Requirements
- At least 5 years of industry experience, ideally in quality assurance with a prior focus on machine learning.
- Strong understanding of machine learning algorithms, supervised and unsupervised modeling, deep learning, computer vision, natural language processing, and generative AI.
- Proficiency in Python and/or Swift or comparable languages.
- Experience preparing and collecting evaluation data, ensuring data quality, and developing prototypes and tools.
- Experience using LLMs and generative AI frameworks to automate model evaluation and optimize test execution.
- Strong analytical, methodical, problem-solving, presentation, and communication skills.
- Bachelor’s degree in Computer Science, Data Science, Applied Mathematics, or a similar field, with understanding of SQA methodologies, machine learning, and software engineering.
- Preferred familiarity with subjective and objective image and video quality evaluation, color science, SQA methodologies, software engineering, iOS, and macOS.
About Apple
Apple designs and sells consumer electronics, software, and services for consumers and professionals worldwide, including iPhone, Mac, iPad, Apple Watch, and AirPods, plus platforms like iOS/macOS and services such as the App Store, iCloud, Music, and TV+. Its business combines device sales with services and subscriptions and in-house silicon design. Founded in 1976, Apple is headquartered in Cupertino, California, and trades on NASDAQ as AAPL.