over 1 year ago
Base Salary
$175k - $280k/yr
Responsibilities
- Optimize Sesame’s serving layer for LLM, speech, and vision models.
- Partner with ML infrastructure and training engineers to build a fast, cost-effective, accurate, and reliable serving layer.
- Modify and extend LLM serving frameworks such as vLLM and SGLang for high-performance model serving.
- Work with the training team to identify ways to produce faster models without sacrificing quality.
- Improve inference performance using in-flight batching, caching, and custom kernels.
- Reduce model initialization times without sacrificing quality.
Requirements
- Expertise in a differentiable array computing framework, preferably PyTorch.
- Expertise optimizing machine-learning models for reliable serving at high throughput and low latency.
- Significant systems programming experience, including work on high-performance server systems or complex PyTorch and vLLM codebases.
- Significant performance engineering experience, such as bottleneck analysis in large-scale server systems or profiling low-level systems code.
- Current knowledge of model-serving optimization techniques.
- Preferred familiarity with high-performance LLM serving, including vLLM or SGLang deployment and internals.
- Preferred experience with a public cloud platform such as GCP, AWS, or Azure.
- Preferred experience deploying and scaling cloud inference workloads using Kubernetes, Ray, or similar tools.
- Track record of leading complex multi-month projects independently is preferred.
- Willingness to learn new things and work across multiple roles is preferred.
Benefits
- Full-time employee benefits include a 3.5% maximum employer 401(k) match.
- 100% employer-paid health, vision, and dental benefits are provided for the employee and dependents.
- Unlimited paid time off and sick time are provided.
- An employer-matched medical flexible spending account is available up to $1,650 per year.
- Guardian Employee Assistance Program is available.
- Competitive stock options provide an opportunity to share in the company’s success.
- Benefits do not apply to contingent or contract workers.