2 months ago
Sŏngnam, Korea, SouthMid Level
Responsibilities
- Optimize LLM inference latency, throughput, memory efficiency, and cost across real-world service environments.
- Develop and optimize GPU- and accelerator-based LLM inference engines and runtimes.
- Research and apply serving techniques including speculative decoding and prefill-decode disaggregation.
- Apply quantization, pruning, distillation, compiler optimization, kernel optimization, and custom operator techniques.
- Optimize LLM execution across mobile, embedded, edge, CPU, GPU, and NPU environments.
- Analyze GPU and runtime bottlenecks using profiling and benchmarking tools across hardware and inference backends.
- Design serving architectures that balance model quality, inference performance, and cost for specified prompts and service requirements.
Requirements
- At least 3 years of experience in LLM, machine learning infrastructure, or inference optimization.
- Experience developing an LLM inference engine or AI runtime.
- Understanding of GPU architecture, CUDA programming, or parallel computing.
- Understanding of quantization, model compression, or compiler optimization techniques.
- Experience with deep learning frameworks such as PyTorch, ONNX, or TensorRT.
- Proficiency in Python and/or C/C++ with strong software engineering capabilities.
- Strong performance analysis, problem-solving, and collaboration skills.
- Preferred experience includes LLM serving frameworks, CUDA or Triton kernels, modern inference optimization, NVIDIA or AMD GPU optimization, edge/on-device LLM optimization, Kubernetes-based AI serving, distributed training, open-source contributions, or relevant AI/ML systems research.
Benefits
- A three-month probationary period may apply.
- The interview process includes document screening, a coding/assignment test, video and in-person or video interviews, and offer negotiation; the process may change based on scheduling and progress.
- Veterans and candidates protected under applicable employment laws are given preference.
- A reference check may be conducted with the candidate's consent after the interview process.
Tech Stack
Categories
About 42dot
42dot builds software-defined vehicle and autonomous mobility platforms for automakers and mobility service providers. Its products include AI-driven vehicle software, data-based update services, and core platform components such as API gateways and identity/access management to connect services securely. The privately held company is headquartered in Seongnam-si, Gyeonggi-do, and focuses on transportation and automotive markets.