2 days ago
Responsibilities
- Build and improve model training services covering pre-training, post-training, and distillation.
- Optimize distributed training compute, memory, communication, kernels, and execution across large GPU and TPU systems.
- Diagnose complex performance problems, implement optimizations, and validate improvements.
- Lead technical work across teams and communicate complex technical tradeoffs.
- Design training APIs and frameworks, including trainer abstractions, configuration systems, and low-level training primitives.
- Contribute to optimization and training-algorithm research, including new optimizers, hyperparameter transfer, and scaling-law studies.
Requirements
- 8+ years of software engineering or ML systems experience, including several years of hands-on distributed training on hundreds or more accelerators.
- Strong understanding of modern model architectures, particularly transformers, including model computation, memory requirements, and training-cost implications.
- Hands-on experience with distributed training optimization using PyTorch Distributed, Megatron-Core, JAX/XLA, or equivalent systems.
- Strong programming skills in Python plus C++, CUDA, Triton, or equivalent performance-oriented technologies.
- Ability to independently diagnose complex performance problems, implement optimizations, and validate improvements.
- Proven record of leading technical work across teams and clearly communicating complex tradeoffs.
- BS, MS, or PhD in Computer Science, Machine Learning, or a related field, or equivalent practical experience.
- Preferred experience with mixture-of-experts, expert parallelism, long-context models, advanced attention mechanisms, high-performance kernels, compilers, GPU and TPU architecture, distributed communication, and low-precision training such as FP8.
- Preferred contributions to relevant open-source projects, experience designing training frameworks, optimization and training-algorithm research, and experience training recommendation, multimodal, or diffusion models at scale.
Categories
About Apple
Apple designs and sells consumer electronics, software, and services for consumers and professionals worldwide, including iPhone, Mac, iPad, Apple Watch, and AirPods, plus platforms like iOS/macOS and services such as the App Store, iCloud, Music, and TV+. Its business combines device sales with services and subscriptions and in-house silicon design. Founded in 1976, Apple is headquartered in Cupertino, California, and trades on NASDAQ as AAPL.