3 days ago
Responsibilities
- Develop training recipes for pre-training, fine-tuning, reinforcement learning, model distillation, and post-training.
- Build and operate large-scale model-training systems across hundreds or thousands of GPUs and TPUs.
- Apply parameter-efficient fine-tuning, preference optimization, quantization-aware training, and other advanced training techniques.
- Diagnose loss spikes and numerical instability and select precision, parallelism, and optimizer configurations for reliable model convergence.
- Optimize training performance across GPU and TPU architectures, memory hierarchies, matrix units, and high-bandwidth memory.
- Set technical direction for Apple’s model-training practices and lead initiatives spanning multiple teams.
- Profile and optimize training workloads using specialized performance-analysis tools.
- Collaborate with Apple researchers and ML engineers to train and ship models meeting Apple’s quality and privacy standards.
Requirements
- At least 8 years of experience in large-scale model training, including reinforcement learning, from research through production.
- At least 5 years of experience building and operating large-scale model-training systems in production.
- Experience with supervised fine-tuning, LoRA-based parameter-efficient fine-tuning, teacher-student distillation, and related techniques.
- Hands-on experience with at least one of Megatron-LM, DeepSpeed, TorchTitan, NeMo, AXLearn, or MaxText.
- Deep expertise in PyTorch or JAX, including distributed-training capabilities.
- Strong programming skills in Python or C++.
- Experience with low-precision training, numerical issues, and data, tensor, pipeline, or expert parallelism.
- Strong understanding of GPU and TPU architecture and experience with NCCL, NVLink, InfiniBand, or TPU ICI.
- Experience with Nsight Systems, PyTorch Profiler, or XProf for profiling and optimization.
- Experience building reinforcement-learning or post-training systems for large models.
- Strong collaboration, written communication, verbal communication, and cross-team technical leadership skills.
- Open-source contributions to training frameworks or publications at venues such as NeurIPS, ICML, or ICLR are preferred.
- BS, MS, or PhD in Computer Science, Computer Engineering, or equivalent practical experience.
Categories
About Apple
Apple designs and sells consumer electronics, software, and services for consumers and professionals worldwide, including iPhone, Mac, iPad, Apple Watch, and AirPods, plus platforms like iOS/macOS and services such as the App Store, iCloud, Music, and TV+. Its business combines device sales with services and subscriptions and in-house silicon design. Founded in 1976, Apple is headquartered in Cupertino, California, and trades on NASDAQ as AAPL.