2 days ago
Base Salary
$184k - $257k/yr
Responsibilities
- Serve as technical owner and domain expert for a GenAI inference framework area such as serving runtime, distributed inference, graph execution and compilation, or PyTorch integration.
- Lead ambiguous multi-quarter programs across technical design, implementation, testing, CI, rollout, and production hardening.
- Design, implement, and ship generative AI inference framework features and MTIA integration layers.
- Profile and optimize production inference performance across frameworks, runtimes, compilers, and kernels.
- Build distributed inference capabilities including hierarchical KV caching, cross-host transfers, disaggregated prefill and decode, and parallelism strategies.
- Optimize graph capture and replay, host-side latency, memory allocation and placement, and production throughput.
- Enable frontier models on MTIA and validate accuracy against GPU baselines.
- Own accuracy, stability, latency, throughput, and regression-test quality for the technical area.
- Partner with kernel, compiler, and silicon teams on hardware/software co-design.
- Mentor engineers, conduct design reviews, and communicate technical decisions across teams.
Requirements
- Bachelor’s degree in Computer Science, Computer Engineering, or a relevant technical field, or equivalent practical experience.
- 8+ years of professional experience in systems software, ML infrastructure, performance engineering, or compiler/framework development.
- 5+ years of hands-on experience with generative AI inference or training frameworks, or production model-serving systems.
- Proficiency in Python, C++, or Rust, including low-level systems code and performance-critical paths.
- Experience optimizing GenAI inference, including prefill and decode, KV-cache management, batching, scheduling, latency, and throughput.
- Hands-on experience with deep learning framework internals such as operator registration and dispatch, eager execution, autograd, graph capture, and compilation.
- Experience with graph-mode execution, host-side latency reduction, memory allocation, and memory placement.
- Experience leading technical design and end-to-end delivery of frameworks or infrastructure across team boundaries, including testing and CI strategies.
- Experience working across framework, compiler, runtime, and hardware layers and root-causing cross-component issues.
- Experience debugging numerical accuracy issues and maintaining accuracy and stability standards.
- Preferred experience with LLM serving stacks, continuous batching, paged or chunked attention, speculative decoding, prefix caching, disaggregated inference, and mixture-of-experts systems.
- Preferred experience with long-context inference, AI accelerator platforms, new hardware backend bring-up, production profiling and tracing, distributed inference or training, RDMA, low-precision numerics, quantization, and PyTorch compilation technologies.
- Preferred familiarity with TorchDynamo, FX IR, Torch Inductor, dynamic shapes, Triton, AOT I, and custom operator integration.
- Preferred experience implementing responsible AI practices and using AI tools, prompt/context engineering, or agent orchestration to improve workflows.
Benefits
- Annual base salary of $183,997/year to $257,000/year, plus bonus, equity, and benefits.
Categories
About Meta
Meta builds social platforms and communication apps—including Facebook, Instagram, WhatsApp, and Messenger—and develops AR/VR hardware and software such as Quest to power immersive computing. It monetizes primarily through advertising tools for businesses, with additional revenue from devices and services, and operates a massive global infrastructure. Founded in 2004 and headquartered in Menlo Park, California, Meta Platforms, Inc. is a public company traded on Nasdaq under the ticker META.