3 hours ago
Base Salary
$250k - $300k/yr
Responsibilities
- Optimize realtime machine learning inference using modern serving frameworks and model-acceleration techniques.
- Profile and improve high-performance inference systems running on NVIDIA GPUs.
- Build distributed, multi-GPU and multi-node serving infrastructure capable of handling thousands of concurrent connections.
- Use Kubernetes, Ray, and custom load balancing to scale and operate inference workloads reliably.
- Take models from research through containerization, serving optimization, and stable production deployment.
- Own ambiguous performance, latency, throughput, and reliability problems from investigation through implementation.
Requirements
- Deep understanding of inference-serving frameworks and techniques such as vLLM or TRT-LLM.
- Hands-on experience with quantization, distillation, caching strategies, continuous batching, paged attention, and speculative decoding.
- Proficiency in C++, CUDA, Rust, or highly optimized Python, including the ability to profile and optimize code for NVIDIA GPUs.
- Experience with Kubernetes, Ray, custom load balancing, multi-GPU or multi-node inference, and high-concurrency systems.
- Non-trivial systems programming projects, major inference-engine open-source contributions, or deep-dive technical write-ups.
- PhD in computer science, physics, or mathematics, or equivalent practical experience building backend or machine learning systems.
- Ability to work independently in ambiguous environments and take systems from research through reliable production operation.
Benefits
- Full-time position with a base salary of $250,000-$300,000 plus bonus, equity, and benefits.
- Relocation assistance is offered.
- In-person work in the Mountain View office.
Tech Stack
Categories
About Inworld
Inworld builds real-time voice AI and agent infrastructure for developers, including text-to-speech, speech-to-speech, speech recognition, and LLM routing delivered via APIs and SDKs. Customers use it to create interactive characters and agentic experiences in games, apps, and virtual worlds; revenue comes from usage-based and enterprise licensing. Founded in 2021 and headquartered in Mountain View, California, the company is privately held and works with game studios and large technology firms.