2 months ago
Base Salary
$263k - $355k/yr
Responsibilities
- Define architecture, interfaces, and the roadmap for distributed AI inference runtime capabilities.
- Enable new model architectures through operator support, production validation, and optimization of scheduling, batching, model execution, memory management, and KV-cache efficiency.
- Profile bottlenecks and develop optimized kernels and data-movement paths across compute, memory, networking, and framework integration.
- Evaluate inference techniques and build benchmarking, regression, validation, and safe-rollout systems.
- Partner with cloud, framework, compiler, hardware, research, and product teams; lead technical reviews and mentor engineers.
- Establish performance-engineering practices and help define and operate production inference workloads.
Requirements
- 8+ years of experience, or equivalent demonstrated impact, in ML systems, high-performance systems, compilers, kernel development, or production AI inference.
- Deep understanding of AI inference, including model execution, attention, Mixture-of-Experts, batching, prioritization, and KV-cache behavior.
- Strong programming skills in C++, Rust, Python, or a comparable language, with knowledge of concurrency, parallel programming, memory resources, and data movement.
- Proven ability to profile, debug, and optimize performance across kernels, runtimes, frameworks, operating systems, and hardware.
- Experience with inference schedulers, cache managers, batching systems, or disaggregated and distributed execution paths is preferred.
- Experience optimizing kernels with accelerator programming tools, assembly, or intrinsics, including attention, matrix multiplication, operator fusion, and low-precision execution is preferred.
- Familiarity with model parallelism, collective communication, high-performance networking, compilers, or graph optimization is preferred.
- Contributions to open-source ML runtimes, frameworks, compilers, or kernel libraries are preferred.
Benefits
- Salary range of $262,700-$355,400 per year, with additional total-reward package details shared during recruitment.
- Hybrid working, with team-specific patterns and role details provided during the application process.
- Collaborative work with AI Platforms, cloud, framework, compiler, hardware, research, and product teams.
- Recruitment adjustments and accommodations are available upon request.
Categories
About Arm
Arm’s foundational technology is defining the future of computing. A future built by the greatest technology ecosystem in the world. A future built on Arm. Arm is everywhere technology matters. Technology matters everywhere. Together, we’ll power every technology revolution moving forward, including cloud computing, automotive and autonomous systems, IoT, the metaverse, and beyond. Changing the world. Again. On Arm.