1 day ago
Base Salary
$186k - $287k/yr
Responsibilities
- Design and implement high-performance Python, C++, CUDA, and Triton code for vLLM-based inference systems, GPU kernels, and numerical computing components.
- Architect, develop, and optimize LLM inference algorithms including quantization, sparsification, speculative decoding, paged attention, KV cache optimization, batching, scheduling, and memory management.
- Profile, benchmark, and model NVIDIA GPU performance using tools including Nsight Systems, Nsight Compute, CUPTI, and PyTorch Profiler.
- Own kernel-level tuning, runtime optimization, distributed inference, multi-GPU serving, tensor parallelism, pipeline parallelism, and large-scale deployment performance.
- Evaluate, prototype, and integrate state-of-the-art LLM serving techniques from research and open-source communities.
- Set engineering standards, conduct peer reviews, and guide testing and benchmarking for performance-critical code.
- Contribute to upstream vLLM and other open-source ML infrastructure through pull requests, issue triage, design discussions, and release readiness support.
- Identify opportunities to improve serving efficiency, hardware utilization, and model acceleration through emerging LLM systems and GPU computing techniques.
Requirements
- Master’s degree in Computer Science, Computer Engineering, or a related field plus three years of experience in the offered or related role, or a bachelor’s degree in those fields plus five years of experience.
- At least three years of experience with high-performance Python and Modern C++ development, NVIDIA GPU performance analysis and profiling using Nsight or similar tools, and GPU memory architecture and optimization for deep learning workloads.
- At least two years of experience making open-source pull requests to PyTorch or vLLM; optimizing CUDA, Triton, or CUTLASS kernels; optimizing LLM inference through quantization, KV caching, or batching; distributed computing with inter- and intra-GPU communication protocols; and leading technical design and reviewing performance-critical code.
Benefits
- Telecommuting is permitted within normal commuting distance of the Red Hat office in Boston, MA.
- The position may be eligible for bonus, commission, and/or equity in addition to annual salary.
- Red Hat supports flexible work environments ranging from in-office and office-flex to fully remote, depending on role requirements.
- Red Hat provides reasonable accommodations to applicants with disabilities and is an equal opportunity workplace.
Tech Stack
Categories
About Red Hat
Red Hat builds enterprise open-source software, including Red Hat Enterprise Linux, OpenShift for Kubernetes, and Ansible automation, sold via subscriptions with support, training, and consulting. It serves large enterprises and public-sector organizations running hybrid and multi-cloud infrastructure and modern application platforms. Founded in 1993 and headquartered in Raleigh, North Carolina, Red Hat operates as an independent subsidiary of IBM.