15 hours ago
Tel Aviv-Yafo, IsraelSenior
Responsibilities
- Develop and optimize compute kernels for a custom ML accelerator targeting production LLM inference performance.
- Implement and validate decoder-only and mixture-of-experts LLM architectures from PyTorch model definitions through distributed execution on custom hardware.
- Integrate custom accelerator backends with vLLM and PyTorch, including scheduler extensions, memory management, and model parallelism.
- Build and maintain model-correctness test infrastructure across CPU, GPU, simulator, and hardware targets.
- Profile inference workloads, identify bottlenecks, and improve latency and throughput from simulation through hardware bringup.
- Own software features from design through implementation, testing, and broader-stack integration.
- Contribute to CI/CD pipelines that detect model and kernel correctness or performance regressions.
- Mentor engineers, lead design reviews, and help raise the team’s engineering standards.
Requirements
- Bachelor’s degree in computer science or equivalent.
- 7+ years of full software development lifecycle experience, including coding standards, code reviews, source control, build processes, testing, and operations.
- Knowledge of machine learning and LLM fundamentals, including transformer architecture, training and inference lifecycles, and optimization techniques.
- Knowledge of computer architecture, operating systems, and parallel computing.
- Strong proficiency in C/C++ and strong Linux systems knowledge.
- Experience developing compute kernels for GPUs, DSPs, or custom accelerators.
- Proven ability to own and deliver complex software features end-to-end.
- Preferred experience with JAX, PyTorch, vLLM, SGLang, Dynamo, TorchXLA, and TensorRT.
- Preferred experience deploying LLMs in production on GPUs, Neuron, TPU, or other AI acceleration hardware, or developing CUDA or other ML/low-level kernels.
- Preferred familiarity with speculative decoding, KV cache optimization, distributed systems, collective communication, RDMA, high-speed interconnect programming, hardware simulation, and model validation workflows.
- Preferred use of LLMs or code-generation agents in daily AI-assisted development workflows.
Benefits
- The employer provides workplace accommodations and application or hiring-process support for candidates with disabilities.
Categories
About Amazon
Amazon builds and operates a global e-commerce marketplace, logistics network, and consumer devices, and provides cloud computing via AWS for businesses and developers. The company earns revenue from online retail, third‑party seller services, subscriptions like Prime, advertising, and AWS usage. Founded in 1994 and headquartered in Seattle, it is publicly traded on NASDAQ (AMZN) and serves customers in dozens of countries.