Senior Software Engineer I - AI Inference Data Plane
DigitalOceanabout 3 hours ago
Base Salary
$139k - $174k/yr
Responsibilities
- Act as a technical leader on the team, driving the design and delivery of data plane components.
- Architect and refine system design proposals for a multi-tenant AI inference cloud ecosystem.
- Implement and optimize distributed inference hosting techniques.
- Collaborate with Product Managers and other teams to align technical roadmaps with customer needs.
- Build on Kubernetes-native distributed inference frameworks to enhance performance.
- Solve distributed-systems problems unique to LLM serving.
- Contribute to open-source projects and represent DigitalOcean in the community.
- Coach and mentor junior engineers to foster technical excellence.
- Maintain and operate critical, high-scale services using observability tools.
Requirements
- Hands-on experience hosting large language or multimodal models using inference engines.
- Familiarity with distributed inference serving frameworks like llm-d or NVIDIA Dynamo.
- Understanding of cluster-scale serving challenges and optimization techniques.
- Proficiency in GoLang or Python and familiarity with gRPC.
- Proven experience shipping customer-facing software products in a high-scale environment.
- Experience integrating and building with open-source software.
Benefits
- Access to resources for career development, including reimbursement for conferences and training.
- Flexible time off policy and Employee Assistance Program.
- Competitive salary with potential bonuses and equity compensation.
- Access to LinkedIn Learning's 10,000+ courses for continued growth.