Member of Technical Staff (AI Inference Engineer)
Perplexity3 months ago
Palo Alto, CA, USA +2 moreMid Level / Senior
H1B Sponsor
Base Salary
$220k - $485k/yr
Responsibilities
- Support transformer-based retrieval, text-generation, and multimodal models in the inference infrastructure.
- Migrate in-house CUDA kernels to NVIDIA's CuTe DSL for improved portability.
- Develop an internal Rust-based inference server to enhance performance.
- Profile and fix performance bottlenecks across the system.
- Build dashboards and alerts for system reliability and observability.
Requirements
- Deep experience with GPU programming and performance work (CUDA, Triton, or similar).
- Understanding of modern LLM architectures and production deployment.
- Experience in building and operating production distributed systems.
- Proficiency in Rust, Python, and CUDA/CuTeDSL.
- 3+ years of professional software engineering experience in ML inference or high-performance systems.