Runtime Engineer
Normal Computing CorporationBase Salary
$200k - $300k/yr
Responsibilities
- Own the runtime architecture, including device management, memory allocation, host-device transfer, executable loading, and command submission.
- Abstract simulators, FPGA prototypes, silicon, and future hardware generations behind a stable interface.
- Manage persistent hardware state such as resident model weights between requests.
- Optimize runtime overhead for LLM inference using end-to-end latency and throughput measurements.
- Define interfaces with the compiler, driver, and firmware layers.
- Set technical direction, review dependent designs, and collaborate with compiler and serving engineers.
Requirements
- Experience building runtimes, drivers, or hardware abstraction layers for accelerators or other non-standard hardware.
- Strong systems programming experience in C++ or Rust, including memory management, concurrency, and low-level performance.
- A track record of owning complex systems components end to end and driving technical decisions across teams.
- Understanding of LLM inference workloads and runtime requirements.
- Comfort developing software for evolving hardware from simulation through bring-up.
- Preferred experience with IREE's HAL, CUDA or ROCm runtime internals, in-memory or non-von-Neumann hardware, inference servers such as vLLM, or open-source runtime and systems infrastructure.
Benefits
- The role works across New York City, Silicon Valley in Mountain View, and London.
- The company provides equal employment opportunity and reasonable accessibility accommodations.
About Normal Computing Corporation
At Normal, we're rewriting AI foundations to advance the frontier of reasoning and reliability in the physical world. We are tackling problems across semiconductors and industrials with a mix of interdisciplinary approaches across the full stack: from probabilistic software infrastructure and algorithms to hardware and physics, enabling AI that can reason and understand its own limits. We understand that our technology is only as powerful as the people behind it. Every employee drives significant impact within our products, often working directly with customers and embedding across our tightly-knit team. Our team members are driven by curiosity and passion for solving some of the most challenging problems in the world of atoms. Normal was founded in 2022 by engineers and scientists that pioneered industry-leading Physics + ML tools for next-gen AI at Google Brain and Google X.