10 months ago
Remote, United States or Remote, CanadaSenior
Base Salary
$180k - $270k/yr
Responsibilities
- Build and ship an LLM-focused inference platform using distributed inference, multi-node deployment, high-performance networking, distributed KV-cache management, and high-throughput batch processing.
- Improve operational excellence through request-to-kernel observability, multi-cloud deployments, autoscaling, and cold-start optimization.
- Collaborate with kernels and GenAI teams to integrate kernel, serving, and cluster optimizations for high application performance.
- Build Helm charts and Kubernetes operators for simple, effective, and maintainable deployments.
- Create durable, reusable software tools and libraries used across teams and functions.
Requirements
- 5+ years of backend engineering experience.
- Experience with Kubernetes and operating owned services.
- Experience building reusable software tools and libraries.
- Experience with machine learning technologies and use cases.
- Bachelor's degree in a relevant field with 4+ years of software engineering or related experience, or a master's degree with 3+ years, or a PhD with 2+ years.
- 2+ years of work experience with a programming language such as Python, Mojo, Rust, or C++.
- Helpful qualifications include high-performance computing or networking experience, large-scale ML inference infrastructure experience, and familiarity with Go.
- Strong collaboration, creativity, curiosity, and alignment with company cultural values.
Benefits
- Candidates may work from the Los Altos, California office or remotely from home in the US or Canada.
- New-hire onboarding is conducted in person in the Los Altos, California office.
- Benefits may include healthcare coverage, retirement and savings programs, employee stock purchase opportunities, paid time off, wellbeing resources, family support programs, and learning and development opportunities.
- The company offers RSU grants, annual target bonuses, equity, and other benefits.
- Regular team onsites and local meetups are provided, with expected travel 2-4 times per year.
- The role is advertised for US and Canadian candidates.
Tech Stack
Categories
About Modular
Modular builds an AI developer platform for training and especially inference/serving, centered on the MAX runtime and the Mojo programming language. Its tools accelerate and deploy models from frameworks like PyTorch and TensorFlow on CPUs and GPUs, for teams running on cloud or on‑prem infrastructure. Founded in 2022, the company operates remote‑first with an office in Los Altos, CA, and sells a commercial platform and enterprise support to organizations productionizing generative and classical ML.