Member of Technical Staff (AI Inference Engineer)
Perplexity6 months ago
London, United KingdomMid Level
Responsibilities
- Support transformer-based retrieval, text-generation, and multimodal models in the inference infrastructure, including weight loading, request scheduling, KV-cache management, and API Gateway support.
- Port in-house CUDA kernels to NVIDIA’s CuTe DSL for current GB200 deployment and future Vera Rubin rack portability.
- Develop the internal Rust-based inference server to improve serving performance and scalability.
- Profile and resolve bottlenecks across network ingress, continuous batching, and interleaved GPU kernels.
- Build dashboards, alerts, and automated remediation for reliability and observability.
- Respond to production incidents and incorporate lessons learned into the system.
Requirements
- At least 3 years of professional software engineering experience with meaningful work on ML inference or high-performance systems.
- Deep experience with GPU programming and performance optimization, using CUDA, Triton, CUTLASS, or similar technologies.
- Understanding of modern LLM architectures and production model serving and inference optimization techniques.
- Experience building and operating production distributed systems under real load, ideally performance-critical systems.
- Ability to work across Rust, Python, and CUDA/CuTe DSL.
- Familiarity with at least one deep learning framework: PyTorch, JAX, or TensorFlow.
- Understanding of GPU architectures, including memory hierarchy, warp scheduling, and tensor cores.
- Nice-to-have experience with PyTorch internals, torch.compile, custom operators, NCCL, NVLink, InfiniBand, RDMA libraries, model/tensor parallelism, quantization, profiling tools, and Kubernetes.
- Ability to own problems end-to-end and work independently in a fast-moving environment.
Benefits
- Equity may be part of the total compensation package.
Tech Stack
Categories
About Perplexity
Perplexity builds an AI-powered answer engine for web and mobile and an enterprise product, Perplexity Computer, for AI-driven workflows across tools and apps. The company monetizes through individual subscriptions (Perplexity Pro) and enterprise plans. Founded in 2022 and headquartered in San Francisco, it operates as a privately held company.