2 months ago
Base Salary
$209k - $283k/yr
Responsibilities
- Define the architecture, interfaces, and roadmap for distributed AI inference runtime capabilities.
- Enable new model architectures through operator support, production validation, scheduling, batching, model execution, memory management, and KV-cache optimization.
- Profile bottlenecks and develop optimized kernels and data-movement paths across compute, memory, networking, and framework integration.
- Build benchmarking, regression, validation, and safe-rollout systems for inference performance and reliability.
- Partner with cloud, framework, compiler, hardware, research, infrastructure, compute, and product teams.
- Lead technical reviews, mentor engineers, and establish performance-engineering practices.
Requirements
- 5+ years of experience, or equivalent demonstrated impact, in ML systems, high-performance systems, compilers, kernel development, or production AI inference.
- Deep understanding of AI inference, including model execution, attention, mixture-of-experts, batching, prioritization, and KV-cache behavior.
- Strong programming skills in C++, Rust, Python, or a comparable language, with knowledge of concurrency, parallel programming, memory resources, and data movement.
- Ability to profile, debug, and optimize performance across kernels, runtimes, frameworks, operating systems, and hardware.
- Experience with inference schedulers, cache managers, batching systems, or distributed execution paths is preferred.
- Experience optimizing kernels with accelerator programming tools, assembly, or intrinsics is preferred.
- Familiarity with model parallelism, collective communication, high-performance networking, compilers, or graph optimization is preferred.
- Contributions to open-source ML runtimes, frameworks, compilers, or kernel libraries are preferred.
Benefits
- Hybrid working with team-defined patterns, subject to local legal, regulatory, tax, and other considerations.
- Collaborative environment with opportunities to work with world-class teams and contribute to foundational AI inference infrastructure.
Categories
About Arm
Arm’s foundational technology is defining the future of computing. A future built by the greatest technology ecosystem in the world. A future built on Arm. Arm is everywhere technology matters. Technology matters everywhere. Together, we’ll power every technology revolution moving forward, including cloud computing, automotive and autonomous systems, IoT, the metaverse, and beyond. Changing the world. Again. On Arm.