24 days ago
Base Salary
$175k - $265k/yr
Responsibilities
- Diagnose performance problems across the full inference serving path, including orchestration, scheduling, networking, autoscaling, batching, KV-cache, serving frameworks, drivers, runtimes, host systems, and accelerators.
- Isolate issues to specific system layers and owners, fixing problems directly or providing evidence-based handoffs and workarounds.
- Protect throughput, latency, and stability targets by identifying and closing performance gaps between orchestrated workloads and bare metal.
- Assess routing, placement, and shared-resource contention when scaling from individual deployments to fleet operation.
- Capture CPU placement, memory, NUMA locality, and network-path requirements in orchestration configuration.
- Build lightweight telemetry and benchmarks for prototype stacks and convert recurring investigations into playbooks.
- Mentor engineers in the profiling-first investigation method.
Requirements
- At least 8 years of hands-on performance or systems engineering experience, including resolving cross-layer regressions.
- Demonstrated ability to profile unfamiliar, sparsely documented systems and identify performance overhead.
- Deep knowledge of Linux internals, NUMA, CPU pinning, cpusets, PCIe, memory and cache behavior, and network paths.
- Experience running performance-sensitive workloads on Kubernetes and addressing overhead from scheduler defaults, network overlays, and orchestration abstractions.
- Working knowledge of LLM inference serving, including batching, KV-cache, prefill/decode, quantization, and speculative decoding.
- Proficiency in Python and C/C++ for instrumentation, debugging, and code patching across the stack.
- Strong technical writing and judgment regarding ownership and cross-team handoffs.
- Experience with LLM serving frameworks such as vLLM, SGLang, or TensorRT-LLM is preferred.
- Preferred qualifications include speculative, disaggregated, or heterogeneous serving; KV-cache and scheduling optimization; GPU programming and profiling with CUDA, Nsight, or NCCL; pre-production hardware bring-up; Rust; and startup experience.
- Familiarity with AI and agentic tools for investigation, code search, log triage, and instrumentation, with the ability to verify results against real data.
Benefits
- Location is Santa Clara, California or remote within the United States.
- The posting describes an inclusive equal-opportunity workplace and affirmative-action employer.
Tech Stack
Categories
About d-Matrix
d-Matrix builds AI inference computing platforms for data centers, combining custom silicon with systems, networking, and software. Its flagship Corsair platform and JetStream fabric focus on low-latency, energy-efficient generative AI inference at scale. Founded in 2019 and headquartered in Santa Clara, California, the privately held company sells hardware with accompanying software to cloud providers and enterprises deploying large AI models.