Senior Engineer, Inference Data Plane
DigitalOceanabout 3 hours ago
Base Salary
$139k - $174k/yr
Responsibilities
- Act as a technical leader driving the design and delivery of data plane components.
- Architect and refine system designs for a multi-tenant AI inference cloud ecosystem.
- Implement and optimize distributed inference hosting techniques.
- Collaborate with Product Managers and engineering teams to align technical roadmaps.
- Build on Kubernetes-native distributed inference frameworks.
- Solve distributed-systems problems unique to LLM serving.
- Contribute to open-source projects and represent DigitalOcean in the community.
- Coach and mentor junior engineers to foster technical excellence.
- Maintain and operate critical high-scale services.
Requirements
- Hands-on experience hosting large language or multimodal models.
- Familiarity with distributed inference serving frameworks.
- Experience with inference engines like vLLM or TensorRT.
- Understanding of challenges in cluster-scale serving.
- Proven contributions to open-source projects preferred.
- Knowledge of LLM architectures and optimization techniques.
- Expert-level proficiency in GoLang or Python.
- Experience shipping customer-facing software products.
- Open-source mindset with experience in building with open-source software.
Benefits
- Competitive benefits including Employee Assistance Program and flexible time off.
- Career development resources including reimbursement for conferences and access to LinkedIn Learning.
- Equity compensation and potential bonuses based on performance.
- Supportive work environment prioritizing employee well-being.