Runtime Engineer
Normal Computing CorporationBase Salary
$200k - $300k/yr
Responsibilities
- Own the runtime architecture covering device management, memory allocation, host-device transfer, executable loading, and command submission.
- Build the runtime against simulators and FPGA prototypes behind a stable interface that can support future silicon.
- Abstract hardware differences across simulators, FPGA prototypes, silicon, and later hardware generations.
- Manage persistent hardware state such as resident model weights between inference requests.
- Optimize runtime overhead for LLM inference using end-to-end latency and throughput measurements.
- Define interfaces with the compiler, serving layer, driver, and firmware teams.
- Set technical direction, review dependent designs, and collaborate with compiler and serving engineers.
Requirements
- Experience building runtimes, drivers, or hardware abstraction layers for accelerators or other non-standard hardware.
- Strong systems programming skills in C++ or Rust, including memory management, concurrency, and low-level performance.
- A track record of owning complex systems components end to end from design through production and driving technical decisions across teams.
- Understanding of LLM inference workloads and runtime requirements.
- Ability to build software for evolving hardware from simulation through bring-up.
- Experience with IREE’s HAL, CUDA, or ROCm runtime internals is a bonus.
- Experience with in-memory compute, non-von-Neumann hardware, vLLM integration, or open-source runtime and systems infrastructure is a bonus.
About Normal Computing Corporation
At Normal, we're rewriting AI foundations to advance the frontier of reasoning and reliability in the physical world. We are tackling problems across semiconductors and industrials with a mix of interdisciplinary approaches across the full stack: from probabilistic software infrastructure and algorithms to hardware and physics, enabling AI that can reason and understand its own limits. We understand that our technology is only as powerful as the people behind it. Every employee drives significant impact within our products, often working directly with customers and embedding across our tightly-knit team. Our team members are driven by curiosity and passion for solving some of the most challenging problems in the world of atoms. Normal was founded in 2022 by engineers and scientists that pioneered industry-leading Physics + ML tools for next-gen AI at Google Brain and Google X.