5 days ago
Palo Alto, CA, USAStaff+
Base Salary
$200k - $350k/yr
Responsibilities
- Build a context-aware inference runtime integrating context selection, compression, conversation compaction, persistent task state, adaptive context budgets, model selection, and reasoning effort.
- Implement paged KV-cache and prompt-prefix-cache management, cache-aware routing, retention, eviction, offloading, and correctness across changing contexts, permissions, model versions, and adapters.
- Optimize inference with speculative decoding, attention kernels, continuous batching, quantization, chunked prefill, and multi-GPU parallel execution.
- Build model and adapter lifecycle management, multi-adapter serving, zero-downtime hot swapping, and resource scheduling.
- Implement production observability, tenant isolation, overload protection, failure recovery, and safe rollout and rollback.
- Develop portable runtime integrations across NVIDIA and AMD GPU platforms, cloud and on-premises environments, and single- or multi-GPU configurations.
Requirements
- 5+ years of engineering experience, including 3+ years of hands-on experience with GPU systems, distributed inference, or ML serving.
- Production experience with vLLM, SGLang, or TensorRT-LLM.
- Strong Python and C++ skills, including C++17/20, with the ability to read and debug CUDA C++ or HIP kernels.
- Expertise in autoregressive generation, prefill and decoding, attention architectures, numerical precision, speculative decoding, KV and prefix caching, context-window management, continuous batching, quantization, and multi-GPU parallelism.
- Preferred experience with custom GPU kernel development using CUDA, HIP, Triton, or CUTLASS.
- Preferred experience with PyTorch internals, torch.compile, custom operators, mixture-of-experts or disaggregated prefill/decode, LoRA or SLM adapter serving, zero-downtime hot swapping, NVIDIA and AMD Instinct hardware, and production telemetry.
Benefits
- Annual incentive opportunity based on target achievement, pre-IPO stock options, medical, dental, vision, 401(k) with match, paid time off, paid holidays, a paid birthday day off, and other paid leave policies.
- Preferred location is Palo Alto, California, USA.
Categories
About Uniphore
Uniphore builds an enterprise Business AI Cloud that powers multimodal, agentic applications across voice, video, and text for customer-facing operations. Its products span speech recognition/analytics, AI agents, orchestration, and workflow automation delivered as B2B software and services to large enterprises in multiple industries. Founded in 2008 and headquartered in Palo Alto, the company is privately held and operates globally.