2 months ago
Base Salary
$180k - $270k/yr
Responsibilities
- Build and deploy high-throughput, ultra-low-latency inference engines for speech models.
- Optimize performance in real-time streaming environments by managing latency and throughput.
- Collaborate with the ML training and backend infrastructure teams.
- Identify and eliminate hardware bottlenecks in GPU architectures.
- Implement advanced inference techniques and manage continuous audio streams.
Requirements
- Hands-on experience with inference engines for large language models.
- Understanding of latency, throughput, and Time-To-First-Token in streaming environments.
- Experience with continuous batching and KV cache management.
- Deep knowledge of GPU architectures and memory hierarchy.
- Strong communication skills and ability to thrive in fast-paced environments.
Benefits
- Competitive compensation package including base salary, performance bonus, and equity.
- Top-tier healthcare benefits for employees and dependents.
- 401(k) plan with company matching.
- Unlimited PTO and 13 paid holidays.
- 12 weeks of paid parental leave.
- Hybrid office model with a minimum of 3 in-office days per week.
Tech Stack
Categories
AI & MLData Engineering