Research Engineer, Infrastructure, Inference
Thinking Machines Lab3 months ago
Base Salary
$400k - $500k/yr
Responsibilities
- Bring cutting-edge AI models into production alongside researchers and engineers.
- Enable high-performance inference for novel model architectures.
- Design and implement techniques, tools, and architectures that improve performance, latency, throughput, and efficiency.
- Optimize codebases and GPU compute fleets for hardware FLOPs, bandwidth, and memory utilization.
- Extend Kubernetes, Ray, and SLURM for distributed inference, evaluation, and large-batch serving.
- Establish reliability, observability, and reproducibility standards across the inference stack.
- Share technical learnings through documentation, open-source libraries, and technical reports.
Requirements
- Bachelor’s degree or equivalent experience in computer science, engineering, or a similar field.
- Understanding of deep learning frameworks and their underlying system architectures, including PyTorch or JAX.
- Experience with throughput- and latency-optimized inference-serving systems such as SGLang or vLLM.
- Strong engineering, coding, and debugging skills in complex codebases.
- Ability to collaborate across cross-functional teams, take initiative, and work across different technical stacks.
- Preferred: experience training or supporting language models with hundreds of billions of parameters or more.
- Preferred: understanding of distributed compute systems, GPU parallelism, and hardware-aware optimizations.
- Preferred: contributions to open-source ML or systems infrastructure projects such as SGLang, vLLM, PyTorch, Triton, DeepSpeed, or XLA.
- Preferred: track record of improving research productivity through infrastructure or process improvements.
Benefits
- Based in San Francisco, California.
- Generous health, dental, and vision benefits.
- Unlimited PTO.
- Paid parental leave.
- Relocation support as needed.
- Visa sponsorship is available, with the company stating it is committed to working through the visa process for the right fit.
- Evergreen role reviewed continuously, with applicants encouraged to reapply no more than once every 6 months.
Tech Stack
Categories
About Thinking Machines Lab
Thinking Machines Lab develops AI and generative AI software and conducts applied research to help organizations make data-driven decisions. The company builds products and data science solutions for enterprise use cases, pairing foundational models with practical tooling and services across industries. It is privately held and headquartered in San Francisco.