Member of Technical Staff (AI Inference Engineer)

4 months ago

London, United KingdomMid Level

H1B Sponsor

Responsibilities

Support transformer-based retrieval, text-generation, and multimodal models in the inference infrastructure, including weight loading, request scheduling, KV-cache management, and API Gateway support.
Port in-house CUDA kernels to NVIDIA’s CuTe DSL for current GB200 deployment and future Vera Rubin rack portability.
Develop the internal Rust-based inference server to improve serving performance and scalability.
Profile and resolve bottlenecks across network ingress, continuous batching, and interleaved GPU kernels.
Build dashboards, alerts, and automated remediation for reliability and observability.
Respond to production incidents and incorporate lessons learned into the system.

At least 3 years of professional software engineering experience with meaningful work on ML inference or high-performance systems.
Deep experience with GPU programming and performance optimization, using CUDA, Triton, CUTLASS, or similar technologies.
Understanding of modern LLM architectures and production model serving and inference optimization techniques.
Experience building and operating production distributed systems under real load, ideally performance-critical systems.
Ability to work across Rust, Python, and CUDA/CuTe DSL.
Familiarity with at least one deep learning framework: PyTorch, JAX, or TensorFlow.
Understanding of GPU architectures, including memory hierarchy, warp scheduling, and tensor cores.
Nice-to-have experience with PyTorch internals, torch.compile, custom operators, NCCL, NVLink, InfiniBand, RDMA libraries, model/tensor parallelism, quantization, profiling tools, and Kubernetes.
Ability to own problems end-to-end and work independently in a fast-moving environment.

201-500 employees

The most powerful answer engine. Powering curiosity with answers backed by up-to-date sources. This is where knowledge begins.