24 days ago
Base Salary
$155k - $235k/yr
Responsibilities
- Design and build end-to-end test infrastructure that provisions, tests, and tears down homogeneous and heterogeneous inference deployments.
- Translate user stories and acceptance criteria into automated end-to-end tests, including failure scenarios.
- Run tests on GPU and Corsair nodes in CI, schedule access to shared hardware, and maintain test-signal quality.
- Own release strategy for the orchestration platform, Kubernetes tooling, and benchmarking tools, including versioning, branching, promotion, rollback, and release criteria.
- Lead the transition from platform-wide releases to independently released components and define compatibility and upgrade testing.
- Define release gates for functional tests, upgrade and compatibility tests, model accuracy evaluations, and performance benchmarks.
- Partner with QE, product, platform, and infrastructure teams on shared test strategy and improve acceptance criteria.
- Enable feature engineers to write and maintain tests for their own work.
Requirements
- 6+ years of experience in test infrastructure, software engineering in test, release engineering, or a closely related role, including ownership of end-to-end testing or releases for a distributed system.
- Demonstrated experience building test or release automation that improved release reliability or speed.
- Experience operating test and release infrastructure across on-premises clusters and a public cloud, preferably Azure.
- Strong Kubernetes experience, including multi-service workloads, Helm, operators, custom resources, and automated cluster setup and teardown.
- Experience designing CI/CD pipelines with artifact versioning, environment promotion, and rollback.
- Proficiency in Python and pytest or a comparable framework, plus fluency in a second programming language sufficient to read and test relevant code.
- Experience testing shared, limited GPU or accelerator hardware fleets, including scheduling, contention, and hardware faults.
- Ability to make and explain end-to-end testing tradeoffs and build cross-team consensus around testing and release strategy.
- Experience using AI and agentic tools for test writing, failure triage, and searching logs and code while verifying their output.
- Preferred: experience testing or releasing LLM serving systems such as vLLM, SGLang, or TensorRT-LLM, including accuracy evaluations and performance benchmarks.
- Preferred: familiarity with speculative decoding, disaggregated serving, or heterogeneous inference.
- Preferred: hardware-in-the-loop testing across driver, firmware, and hardware versions.
- Preferred: experience with envtest, kuttl, or chainsaw, CRD version migration, and upgrade testing.
- Preferred: experience shipping software to customer-managed on-premises Kubernetes environments, including compatibility matrices and customer install packaging.
- Preferred: experience with canary or progressive rollouts, Rust, or startup environments.
Benefits
- Location is Santa Clara, California or remote in the United States.
- The role works across on-premises clusters and Azure-based infrastructure.
Tech Stack
Categories
About d-Matrix
d-Matrix builds AI inference computing platforms for data centers, combining custom silicon with systems, networking, and software. Its flagship Corsair platform and JetStream fabric focus on low-latency, energy-efficient generative AI inference at scale. Founded in 2019 and headquartered in Santa Clara, California, the privately held company sells hardware with accompanying software to cloud providers and enterprises deploying large AI models.