Senior Engineer 2: Inference Data Plane

about 5 hours ago

Austin, TX, USA

Senior / Staff+

H1B Sponsor

Base Salary

$167k - $209k/yr

Responsibilities

Act as a technical leader on the team, driving the design and development of data plane components.
Architect and refine system design proposals for a high-scale AI inference cloud ecosystem.
Implement and optimize distributed inference hosting using advanced techniques.
Collaborate with Product Managers and other teams to align technical roadmaps with customer needs.
Coach and mentor junior engineers to foster a culture of technical excellence.
Maintain and operate critical, high-scale services using observability tools.

Strong experience with distributed systems, microservices, and infrastructure as code.
Hands-on experience hosting large language or multimodal models using inference engines.
Familiarity with distributed inference serving frameworks like llm-d or NVIDIA Dynamo.
Understanding of GPU-level optimization and interconnect technologies.
Knowledge of common LLM architectures and optimization techniques.
Expert-level proficiency in GoLang or Python and familiarity with gRPC.
Proven experience shipping customer-facing software products in a high-scale environment.
Experience integrating and building with open-source software.

Competitive array of benefits including an Employee Assistance Program and flexible time off policy.
Reimbursement for relevant conferences, training, and education.
Access to LinkedIn Learning's 10,000+ courses for continued growth.
Potential for bonuses and equity compensation based on performance.

GogRPCPython