Member of Technical Staff (AI Inference Engineer)
Perplexity7 months ago
Palo Alto, CA, USA +2 moreMid Level
H1B sponsor
Base Salary
$220k - $485k/yr
Responsibilities
- Support transformer-based retrieval, text-generation, and multimodal models in inference infrastructure.
- Manage weight loading, request scheduling, KV-cache management, and API serving.
- Port in-house CUDA kernels to NVIDIA’s CuTe DSL for current and future GPU platforms.
- Develop the internal Rust-based inference server.
- Profile and optimize network ingress, continuous batching, and GPU kernel interleaving.
- Build dashboards, alerts, and automated remediation for reliability and observability.
- Respond to production incidents and improve systems based on operational learnings.
Requirements
- 3+ years of professional software engineering experience with meaningful work on ML inference or high-performance systems.
- Deep experience with GPU programming and performance optimization, including CUDA, Triton, CUTLASS, or similar technologies.
- Understanding of modern LLM architectures and production inference deployment.
- Experience building and operating production distributed systems under real load.
- Proficiency working across Rust, Python, and CUDA or CuTe DSL.
- Familiarity with at least one deep learning framework: PyTorch, JAX, or TensorFlow.
- Understanding of GPU architectures, including memory hierarchy, warp scheduling, and tensor cores.
- Understanding of LLM inference optimization techniques such as quantization, speculative decoding, and prefill-decode disaggregation.
- Experience with ML compiler or framework internals, distributed GPU communication, low-precision inference, profiling and debugging tools, or Kubernetes is beneficial.
Tech Stack
Categories
About Perplexity
Perplexity builds an AI-powered answer engine for web and mobile and an enterprise product, Perplexity Computer, for AI-driven workflows across tools and apps. The company monetizes through individual subscriptions (Perplexity Pro) and enterprise plans. Founded in 2022 and headquartered in San Francisco, it operates as a privately held company.