Large Language Model Inference System Engineer Graduate (Applied Machine Learning) - 2027 Start
ByteDanceResponsibilities
- Develop the Volcano Ark MaaS large-model inference system.
- Optimize inference performance, cost, and stability across ultra-large-scale heterogeneous inference clusters.
- Reduce inference costs using disaggregated multi-role inference, distributed KV Cache systems, heterogeneous inference, elastic computing, and multi-tenant co-located inference.
- Optimize distributed network communication and inference across multiple nodes when applicable.
Requirements
- Be completing or have recently completed a bachelor’s or master’s degree in computer science or a related discipline.
- Demonstrate strong knowledge of algorithms, design patterns, data structures, operating systems, and computer architecture.
- Be proficient in one or more programming languages such as C++ or Python.
- Understand GPU hardware architecture and be familiar with high-performance computing software stacks such as CUDA.
- Have experience with GPU performance analysis and a strong interest in distributed systems and large-scale heterogeneous inference.
- Preferred: experience optimizing large-model inference systems, including PD disaggregation, KV Cache systems, or multi-node inference.
- Preferred: experience with distributed network communication optimization, distributed communication operator implementation, and RDMA principles.
- Preferred: familiarity with resource orchestration and scheduling frameworks such as Kubernetes and Ray.
- State availability and graduation date clearly and commit to an onboarding date by the end of the year.
Tech Stack
Categories
About ByteDance
ByteDance is a global incubator of platforms at the cutting edge of commerce, content, entertainment and enterprise services - over 2.5bn people interact with ByteDance products including TikTok. Creation is the core of ByteDance's purpose. Our products are built to help imaginations thrive. This is doubly true of the teams that make our innovations possible. Together, we inspire creativity and enrich life - a mission we aim towards achieving every day. At ByteDance, we create together and grow together. That's how we drive impact - for ourselves, our company, and the users we serve. We are committed to building a safe, healthy and positive online environment for all our users. We have over 110,000 employees based in more than 30 countries globally. Join us.