4 hours ago
Base Salary
$236k - $330k/yr
Responsibilities
- Design and develop high-performance LLM inference systems spanning distributed serving, runtime systems, GPU execution, and performance-critical kernels.
- Develop techniques to improve inference latency, generation speed, throughput, memory efficiency, scalability, and cost.
- Explore speculative and parallel decoding, disaggregated inference, adaptive parallelism, batching, scheduling, KV-cache management, quantization, and communication optimization.
- Build adaptive inference systems that optimize execution for new model architectures, hardware platforms, workloads, and deployment environments.
- Apply AI-native approaches including automated profiling, bottleneck identification, configuration search, code generation, experimentation, debugging, and performance tuning.
- Identify high-impact systems problems, develop hypotheses and prototypes, and take promising ideas from research through production.
- Design distributed inference strategies across GPUs and nodes using tensor, sequence, pipeline, data, and expert parallelism.
- Develop multi-model serving, dynamic resource management, model loading and swapping, and workload-aware scheduling.
- Analyze and optimize GPU kernels and operators for attention, mixture-of-experts, communication, and other performance-critical components.
- Explore model-system co-design and model or post-training techniques for more efficient inference.
- Profile and benchmark end-to-end workloads across compute, memory, communication, networking, scheduling, and model execution.
- Collaborate with model researchers, infrastructure teams, and product teams to deploy research innovations in production.
- Open-source and publish innovations through technical blogs and top-tier systems and machine learning conferences.
Requirements
- Bachelor's degree in Computer Science, Electrical Engineering, or a related field; a master's degree or PhD is preferred.
- 5+ years of experience in LLM inference systems, distributed AI systems, GPU systems, or high-performance computing.
- Strong understanding of modern LLM inference architectures and performance tradeoffs for large-scale model serving.
- Hands-on experience with LLM inference and serving frameworks such as vLLM, SGLang, or TensorRT-LLM.
- Experience designing, extending, or optimizing inference runtimes, including scheduling, batching, KV-cache management, distributed execution, parallelism, speculative decoding, or disaggregated serving.
- Strong understanding of GPU architectures and experience with CUDA, Triton, or similar GPU programming environments.
- Experience with performance-oriented libraries and frameworks such as CUTLASS, cuBLAS, or cuDNN.
- Experience profiling and diagnosing end-to-end system performance using Nsight Systems, Nsight Compute, or equivalent tools.
- Ability to independently identify important problems, define technical questions, and drive solutions through ambiguity.
- Ability to work across model, runtime, distributed system, and hardware layers while reasoning about end-to-end performance tradeoffs.
- Experience with AI-native engineering approaches for software development, experimentation, debugging, optimization, or system adaptation is a strong plus.
- Excellent communication and cross-functional collaboration skills across research, engineering, and product teams.
Benefits
- Salary and benefits information is provided through the Snowflake Careers Site for jobs located in the United States.
Tech Stack
SnowflakeTensorFlow
Categories
About Snowflake
Snowflake builds a cloud-native data platform used by enterprises to store, integrate, share, and analyze data across AWS, Azure, and Google Cloud. Its core products span data warehousing, data lakes, data engineering, and governed data sharing, sold via consumption-based subscriptions. Founded in 2012 and publicly traded on the NYSE (SNOW) following a 2020 IPO, Snowflake supports analytics and data application workloads across industries.