5 days ago
Palo Alto, CA, USASenior / Staff+
Base Salary
$200k - $350k/yr
Responsibilities
- Build a context-aware runtime integrating context selection, compression, conversation compaction, persistent task state, model selection, and adaptive reasoning effort.
- Develop paged KV-cache and prompt-prefix-cache management, including routing, retention, eviction, offloading, quantization, and cross-request reuse.
- Implement and optimize speculative decoding, attention kernels, continuous batching, chunked prefill, quantization, and multi-GPU parallel execution.
- Manage model and adapter lifecycles, multi-adapter serving, zero-downtime hot swapping, and resource scheduling.
- Implement production observability, tenant isolation, overload protection, failure recovery, and safe rollout and rollback mechanisms.
- Build portable integrations across NVIDIA and AMD GPUs, cloud and on-premises environments, and single- and multi-GPU configurations.
Requirements
- At least 5 years of engineering experience, including at least 3 years of hands-on experience with GPU systems, distributed inference, or ML serving.
- Production experience with vLLM, SGLang, or TensorRT-LLM.
- Strong Python and C++ skills, including the ability to read and debug CUDA C++ or HIP kernels.
- Hands-on GPU profiling and performance optimization experience.
- Preferred experience with custom GPU kernel development using CUDA, HIP, Triton, or CUTLASS.
- Preferred experience with PyTorch internals, torch.compile, custom operators, mixture-of-experts, or disaggregated prefill/decode.
- Preferred experience with LoRA or SLM adapter serving, zero-downtime hot swapping, and NVIDIA or AMD Instinct hardware in production.
- Preferred experience with telemetry using DCGM, AMD SMI, Prometheus, and Grafana.
Benefits
- Annual incentive opportunity based on target achievement and pre-IPO stock options.
- Medical, dental, vision, and 401(k) with company match.
- Generous paid time off, paid holidays, birthday leave, and other paid leave policies.
- Preferred location is Palo Alto, California, USA.
Categories
About Uniphore
Uniphore builds an enterprise Business AI Cloud that powers multimodal, agentic applications across voice, video, and text for customer-facing operations. Its products span speech recognition/analytics, AI agents, orchestration, and workflow automation delivered as B2B software and services to large enterprises in multiple industries. Founded in 2008 and headquartered in Palo Alto, the company is privately held and operates globally.