Senior Software Engineer I - AI Inference Data Plane
DigitalOceanabout 3 hours ago
Base Salary
$139k - $174k/yr
Responsibilities
- Act as a technical leader on the team, driving the design and delivery of data plane components.
- Architect and refine system design proposals for a multi-tenant AI inference cloud ecosystem.
- Implement and optimize distributed inference hosting techniques.
- Collaborate with Product Managers and other teams to align technical roadmaps with customer needs.
- Build on Kubernetes-native distributed inference frameworks to enhance model serving.
- Solve distributed-systems problems unique to LLM serving.
- Contribute to open-source projects and represent DigitalOcean in the community.
- Coach and mentor junior engineers to foster technical excellence.
- Maintain and operate critical, high-scale services using observability tools.
Requirements
- Hands-on experience hosting large language or multimodal models using inference engines.
- Familiarity with distributed inference serving frameworks.
- Expertise in inference engines like vLLM or alternatives.
- Understanding of challenges in cluster-scale serving.
- Proven track record of contributions to open-source projects.
- Knowledge of common LLM architectures and optimization techniques.
- Expert-level proficiency in GoLang or Python and familiarity with gRPC.
- Experience shipping customer-facing software products in a high-scale environment.
- Open-source mindset with experience integrating and building with open-source software.
Benefits
- Competitive array of benefits including Employee Assistance Program and flexible time off policy.
- Career development resources including reimbursement for conferences and access to LinkedIn Learning courses.
- Equity compensation and potential bonuses based on performance.