Senior Engineer 2: Inference Data Plane

about 5 hours ago

Denver, CO, USA

Senior

H1B Sponsor

Base Salary

$167k - $209k/yr

Responsibilities

Act as a technical leader on the team, driving the design and delivery of data plane components.
Architect and refine system design proposals for a multi-tenant AI inference cloud ecosystem.
Implement and optimize distributed inference hosting using advanced techniques.
Collaborate with Product Managers and other teams to align technical roadmaps with customer needs.
Coach and mentor junior engineers to foster a culture of technical excellence.
Maintain and operate high-scale services, utilizing observability tools to ensure platform health.

Strong experience with distributed systems, microservices, and infrastructure as code.
Hands-on experience hosting large language or multimodal models using inference engines.
Familiarity with distributed inference serving frameworks like llm-d or NVIDIA Dynamo.
Understanding of GPU-level optimization and interconnect technologies.
Knowledge of common LLM architectures and optimization techniques.
Expert-level proficiency in GoLang or Python and familiarity with gRPC.
Proven experience shipping customer-facing software products in a high-scale environment.
Experience integrating and building with open-source software.

Access to resources for career development, including reimbursement for conferences and training.
Flexible time off policy and support for employee well-being.
Competitive salary with potential bonuses and equity compensation.

GogRPCPython