over 1 year ago
Bellevue, WA, USA +2 moreMid Level / Senior
H1B Sponsor
Base Salary
$175k - $280k/yr
Responsibilities
- Turbocharge the serving layer with LLM, speech, and vision models.
- Collaborate with ML infrastructure and training engineers to build an efficient serving layer.
- Modify and extend LLM serving frameworks like VLLM and SGLang.
- Identify opportunities for faster model production without quality loss.
- Implement techniques like in-flight batching and caching to enhance inference speed.
- Reduce model initialization times while maintaining quality.
Requirements
- Expert in a differentiable array computing framework, preferably PyTorch.
- Experience optimizing machine learning models for high throughput and low latency.
- Significant systems programming experience with high-performance server systems.
- Strong performance engineering experience, including bottleneck analysis.
- Up-to-date knowledge of the latest model serving optimization techniques.
Benefits
- 401 (k) max employer match: 3.5% of compensation.
- 100% employer-paid health, vision, and dental benefits for you and your dependents.
- Unlimited PTO and sick time.
- Flexible spending account with employer matching up to $1,650/year.
- Guardian Employee Assistance Program (EAP).
- Opportunity to share in the company's success with competitive stock options.