Head of Inference, Stealth Edge AI Co
Montauk Capital5 months ago
Responsibilities
- Define the inference strategy and architecture for Edge AI.
- Own the inference serving layer end to end.
- Build a credible proof of concept for NVIDIA, cloud providers, and customers.
- Optimize cost per token, GPU utilization, KV-cache management, batching, latency, and throughput.
- Build distributed inference pipelines across multi-GPU, multi-node edge deployments.
- Define quantization strategy and translate inference requirements into infrastructure designs.
- Own observability and reliability SLAs and establish performance baselines.
- Define the software access-layer architecture and oversee integration efforts.
- Represent the company with investors, partners, early customers, and technical stakeholders.
Requirements
- Direct experience building production or near-production inference serving systems at scale.
- Deep expertise in model serving, quantization, GPU utilization, CUDA kernel optimization, batching, KV-cache management, and speculative decoding.
- Experience with vLLM, TensorRT-LLM, Triton Inference Server, SGLang, or equivalent technologies.
- Experience with distributed tracing, latency profiling, alerting, and observability for complex distributed systems.
- Proficiency in C++, CUDA, or Rust.
- Experience scaling systems with Kubernetes, Ray, custom load balancing, and multi-GPU or multi-node inference.
- Experience with model deployment and serving, systems engineering, and technical leadership over teams or outcomes.
- Ability to move quickly in ambiguous startup and 0-to-1 environments and communicate clearly.
Benefits
- Full-time remote role, with NYC preferred and US-based hiring.
- Competitive compensation and equity.
- Studio support, resources, network, and operational expertise from Montauk Capital.
Tech Stack
Categories
About Montauk Capital
Montauk Capital is a privately held investment platform based in New York City that both backs and incubates companies in the “Electron Economy.” The firm runs venture capital and venture studio strategies across electrification, energy storage, industrial software, and transportation, supporting software-enabled energy and industrial systems. Founded in 2023, it targets opportunities from data centers and grid technologies to robotics and supply-chain resiliency, providing capital and hands-on company-building.