10 hours ago
Responsibilities
- Design distributed systems for routing and scheduling LLM inference requests across a global inference-server fleet.
- Develop routing strategies that balance latency, availability, compute efficiency, changing demand, capacity, and failures.
- Design system architectures, specify custom network protocols, and write high-performance server code.
- Build scaling strategies for services used by hundreds of millions of people.
- Evaluate cache-aware request placement, KV-cache reuse, and prefill/decode disaggregation within Private Cloud Compute’s privacy model.
- Own technical decisions from early design through production using benchmarks and operational evidence.
- Collaborate with model serving, infrastructure, security, and SRE engineers to deliver reliable inference-routing capabilities.
Requirements
- At least 3 years of professional experience designing, building, and operating large distributed systems in production.
- Experience with server development in Java, C, C++, C#, Go, or Swift.
- Knowledge of LLM inference serving, including request routing, KV-cache placement and reuse, prefill/decode disaggregation, and latency, throughput, and accelerator-use tradeoffs.
- Experience with network protocols, application-level networking, client-server architectures, request routing, load balancing, or scheduling at scale.
- Experience benchmarking, debugging, and tuning distributed-system performance, with knowledge of Unix system internals.
- Experience with cloud deployment and observability technologies such as Kubernetes.
- Bachelor’s degree in an engineering-related field or equivalent experience.
- Preferred experience building LLM inference-serving or routing systems such as llm-d, NVIDIA Dynamo, vLLM, or SGLang.
- Preferred familiarity with KV-cache-aware routing, prefix caching, prefill/decode disaggregation, and accelerator-capacity placement.
- Preferred experience with TLS, X.509 certificate evaluation, secure protocols, cryptographic protocols, encryption, key management, and privacy-preserving system design such as Oblivious HTTP and remote attestation.
- Preferred experience leading substantial engineering projects from design through production and coordinating across teams.
Tech Stack
Categories
About Apple
Apple designs and sells consumer electronics, software, and services for consumers and professionals worldwide, including iPhone, Mac, iPad, Apple Watch, and AirPods, plus platforms like iOS/macOS and services such as the App Store, iCloud, Music, and TV+. Its business combines device sales with services and subscriptions and in-house silicon design. Founded in 1976, Apple is headquartered in Cupertino, California, and trades on NASDAQ as AAPL.