Research Engineer, Training & Inference

3 months ago

Palo Alto, CA, USAMid Level

H1B Sponsor

Base Salary

$200k - $450k/yr

Responsibilities

Maintain and optimize proprietary reinforcement learning training and serving infrastructure across the full stack.
Maximize reinforcement learning throughput from data generation through model training using sharded multi-node training and inference algorithms.
Optimize inference engines, routers, schedulers, and custom kernels for high-throughput reinforcement learning and low-latency LLM production traffic.
Identify and resolve distributed-cluster performance bottlenecks to improve throughput and memory efficiency for multi-billion-parameter models.
Refactor layers from Python APIs through CUDA kernels to achieve peak performance for foundation model workloads.

Bachelor’s degree in computer science or a related technical field, or equivalent industry experience.
At least 2 years of relevant hands-on industry experience.
Proficiency in Python.
Experience building or maintaining components within PyTorch, JAX, or TensorFlow.
Understanding of distributed training concepts and collective communication primitives such as NCCL, or practical experience deploying and profiling models on GPU-accelerated cloud infrastructure.
Preferred: master’s or PhD in computer science, mathematics, or a related field.
Preferred: 5+ years of relevant hands-on industry experience.
Preferred: proficiency in C++.
Preferred: experience writing or improving kernels with Triton, CuTeDSL, TileLang, CUDA, CUTLASS, or ThunderKittens.
Preferred: experience deploying performant inference at scale using open-source or custom inference engines and routers.
Preferred: experience scaling models with FSDP, Tensor Parallelism, or related sharding techniques on multi-node GPU clusters.
Preferred: experience designing reinforcement learning systems for high-throughput training and asynchronous data sampling.

11-50 employees

We are forging the worlds most advanced mathematical reasoning engine