3 months ago
Base Salary
$195k - $285k/yr
Responsibilities
- Prototype emerging LLM inference use cases for heterogeneous hardware deployments.
- Build proof-of-concept systems for customers, partners, and internal stakeholders.
- Develop and tune custom kernels and operator-level optimizations for throughput and latency.
- Drive quantization, sparsity, and batching strategies for the D-Matrix computational model.
- Build and maintain inference runtimes, serving frameworks, and evaluation tooling.
- Contribute to distributed inference, including tensor and pipeline parallelism, disaggregated prefill/decode, and KV-cache management.
- Provide inference workload insights to hardware architects, firmware teams, and compiler teams.
- Partner with product and business development on customer-facing demonstrations.
- Contribute to technical publications, whitepapers, and open-source projects.
Requirements
- Bachelor’s degree in Computer Science, Electrical Engineering, or a related field and 10+ years of relevant engineering experience, or equivalent demonstrated experience.
- Master’s or PhD in Computer Science, Electrical Engineering, or a related field is preferred, with 6+ years of relevant industry experience.
- Strong proficiency in Python and C/C++.
- Hands-on experience optimizing LLM inference, including attention kernels, KV cache, batching strategies, and INT8, FP8, or INT4 quantization.
- Contributor-level experience with at least one major inference framework, such as vLLM, SGLang, TensorRT-LLM, or ONNX Runtime.
- Familiarity with GPU kernel programming using CUDA or Triton and performance profiling tools.
- Experience with heterogeneous compute deployments, custom silicon or ASIC-based inference, distributed inference, production inference serving at scale, or open-source inference and ML systems projects is preferred.
- Familiarity with speculative decoding, mixture-of-experts routing, long-context serving, and the JAX Scaling Book or equivalent systems-level understanding is preferred.
Benefits
- Competitive compensation, equity, and benefits.
- Work location is Santa Clara, California.
Categories
About d-Matrix
d-Matrix builds AI inference computing platforms for data centers, combining custom silicon with systems, networking, and software. Its flagship Corsair platform and JetStream fabric focus on low-latency, energy-efficient generative AI inference at scale. Founded in 2019 and headquartered in Santa Clara, California, the privately held company sells hardware with accompanying software to cloud providers and enterprises deploying large AI models.