Senior Engineer, Inference Data Plane
DigitalOceanabout 3 hours ago
Base Salary
$139k - $174k/yr
Responsibilities
- Act as a technical leader driving the design and delivery of data plane components.
- Architect and refine system design proposals for a multi-tenant AI inference cloud ecosystem.
- Implement and optimize distributed inference hosting techniques.
- Collaborate with Product Managers and engineering teams to align technical roadmaps.
- Build on Kubernetes-native distributed inference frameworks.
- Solve distributed-systems problems unique to LLM serving.
- Contribute to open-source projects and represent DigitalOcean in communities.
- Coach and mentor junior engineers.
- Maintain and operate critical high-scale services.
Requirements
- Hands-on experience hosting large language or multimodal models.
- Familiarity with distributed inference serving frameworks.
- Experience with inference engines like vLLM or TensorRT.
- Understanding of challenges in cluster-scale serving.
- Proven contributions to open-source projects.
- Knowledge of LLM architectures and optimization techniques.
- Expert-level proficiency in GoLang or Python.
- Experience shipping customer-facing software products.
- Open-source mindset and experience integrating with open-source software.
Benefits
- Access to resources for career development, including conference reimbursements.
- Flexible time off policy and Employee Assistance Program.
- Competitive salary with potential bonuses and equity compensation.
- Access to LinkedIn Learning's extensive course library.