1 day ago
Base Salary
$175k - $265k/yr
Responsibilities
- Diagnose performance problems across orchestration, scheduling, placement, networking, autoscaling, inference internals, runtimes, host operating systems, and accelerators.
- Isolate root causes to specific system layers and owners, fixing issues directly or handing them off with reproducible findings and evidence.
- Protect throughput, latency, and stability targets by identifying gaps between orchestrated workloads and bare-metal performance.
- Assess fleet-scale deployment concerns including routing, placement, and shared-resource contention.
- Capture CPU placement, memory, NUMA locality, and network-path requirements in orchestration configuration.
- Build lightweight telemetry and benchmarks for prototype stacks.
- Convert recurring investigations into playbooks and mentor engineers in a profiling-first performance methodology.
Requirements
- 8+ years of hands-on performance or systems engineering experience, including resolving cross-layer performance regressions.
- Demonstrated ability to profile unfamiliar, sparsely documented systems and identify performance overhead.
- Deep knowledge of Linux internals, NUMA, CPU pinning and cpusets, PCIe, memory and cache behavior, and network paths.
- Experience running performance-sensitive workloads on Kubernetes and addressing overhead from scheduler defaults, network overlays, and orchestration abstractions.
- Working knowledge of LLM inference concepts including batching, KV-cache, prefill/decode, quantization, and speculative decoding.
- Proficiency in Python and C/C++ for instrumentation, debugging, and cross-stack patching.
- Strong technical writing for hardware, firmware, and platform teams, plus sound judgment about ownership and handoffs.
- Fluency with AI and agentic tools for investigation, code search, log triage, and instrumentation drafting, with the ability to verify results against real data.
- Preferred experience with vLLM, SGLang, TensorRT-LLM, speculative decoding, disaggregated or heterogeneous serving, KV-cache management, request and batch scheduling, weight prefetch and residency, GPU programming and profiling, pre-production hardware, or Rust.
- Startup experience or comfort working in an environment with shifting priorities.
Benefits
- Location is Santa Clara, California or remote within the United States.
- The company describes an inclusive, collaborative, and equal-opportunity workplace.
Tech Stack
Categories
About d-Matrix
d-Matrix builds AI inference computing platforms for data centers, combining custom silicon with systems, networking, and software. Its flagship Corsair platform and JetStream fabric focus on low-latency, energy-efficient generative AI inference at scale. Founded in 2019 and headquartered in Santa Clara, California, the privately held company sells hardware with accompanying software to cloud providers and enterprises deploying large AI models.