Senior Engineer II, Serverless Inference
DigitalOcean6 hours ago
Base Salary
$167k - $209k/yr
Responsibilities
- Design and build scalable, multi-tenant services for AI inference and intelligent routing workloads.
- Improve platform resiliency through observability, capacity management, automation, and operational tooling.
- Partner with platform, GPU infrastructure, and product engineering teams to deliver production-grade systems and highly available APIs.
- Contribute to architecture decisions involving traffic management, service orchestration, reliability, and platform scalability.
- Participate in on-call rotations, incident management, and efforts to improve service health and reduce recurring incidents.
- Provide technical leadership by clarifying ambiguous problems and guiding engineers toward execution plans.
- Coach and support junior and mid-level engineers through pairing and constructive feedback.
Requirements
- 7+ years of experience building and operating multi-tenant platforms or distributed backend systems.
- Strong experience operating high-scale distributed services in production.
- Deep understanding of observability, incident management, reliability engineering, capacity planning, and operational automation.
- At least 1 year of hands-on production experience with Go or Golang.
- At least 2 years of experience with Kubernetes.
- Strong understanding of cloud-native architectures, microservices, and distributed systems fundamentals.
- Experience debugging production performance, scalability, and reliability issues.
- Experience tracking inference and infrastructure metrics such as TTFT, TPOT, and GPU utilization.
- Preferred experience with LLM serving architectures and technologies including vLLM, Triton, and TensorRT-LLM.
- Preferred experience with API gateways, traffic routing, service meshes, inference optimization, rate limiting, or workload orchestration.
Benefits
- Hybrid work arrangement.
- Reimbursement for relevant conferences, training, and education.
- Access to LinkedIn Learning courses.
- Employee Assistance Program, local employee meetups, and flexible time off.
- Potential eligibility for performance-based bonuses and equity compensation, including equity grants and an Employee Stock Purchase Program.
Tech Stack
About DigitalOcean
DigitalOcean provides cloud infrastructure and platform services for developers, startups, and small to mid-sized businesses, including virtual machines (Droplets), managed Kubernetes and databases, object/block storage, networking, and GPUs for AI workloads. It operates a usage-based, self-service public cloud with APIs, CLI, and a marketplace to deploy and scale applications. Founded in 2012 and headquartered in Broomfield, Colorado, DigitalOcean is a public company listed on the NYSE.