Research Engineer - Inference
Prime Intellect7 hours ago
Responsibilities
- Extend prime-rl’s vLLM integration to support new model architectures in RL training.
- Improve rollout throughput for long-horizon agents through concurrency tuning and KV-cache reuse.
- Optimize multi-node MoE inference, including expert parallelism and prefill/decode tradeoffs.
- Reduce trainer-to-inference weight-update overhead using sharding-aware transfer and communication overlap.
- Build on FP8, evaluate lower precision, and use trainer/sampler mismatch checks to identify numerical regressions.
- Collaborate with model researchers on architecture choices that balance model quality and inference cost.
Requirements
- Experience building or optimizing GPU inference systems and understanding inference-engine internals.
- Strong Python skills and experience with PyTorch or a comparable ML framework.
- Ability to profile end-to-end workloads and convert measurements into verified performance improvements.
- Experience debugging numerical or integration issues in machine-learning systems.
- Preferred: contributions to vLLM or another inference engine.
- Preferred: CUDA or Triton kernel experience, particularly for low-precision inference.
Benefits
- Flexible work arrangements with remote or in-person work available at the San Francisco offices.
- Visa sponsorship and relocation assistance for international candidates.
- Quarterly team off-sites, hackathons, conferences, and learning opportunities.
- Opportunity to work with a mission-driven team building frontier AI infrastructure.
Categories
About Prime Intellect
Prime Intellect builds a full-stack platform for training frontier AI models, giving enterprises access to distributed compute and agentic training infrastructure to develop their own systems. The company operates as a platform and open research lab, with customers leveraging its infrastructure rather than building it in-house. Headquartered in San Francisco and privately held, it has raised a Series A round backed by investors including Founders Fund and NVIDIA.