1 day ago
London, United KingdomSenior
Responsibilities
- Build self-hosted LLM serving and model routing for air-gapped and constrained deployments.
- Own LLM packaging, registration, deployment, health checks, upgrades, rollback, and retirement.
- Evaluate and integrate serving platforms such as KServe, including inference services, runtimes, model loading, and rollout controls.
- Define and implement service levels for availability, quality, and performance metrics such as time to first token, inter-token latency, and tail latency.
- Optimize throughput, GPU utilization, memory use, and cost per token through batching, caching, quantization, speculative decoding, and parallelism.
- Build autoscaling, admission control, capacity management, observability, and explainable routing around GPU and serving signals.
- Work with Research and Post-Training teams to move models from evaluation into supported production routes.
- Help determine how the LLM serving capability should grow and shape the team over time.
Requirements
- Production experience operating inference engines such as vLLM or SGLang, or serving platforms such as KServe.
- Experience building or operating model routing across multiple providers while balancing cost, quality, latency, failure, and policy.
- Strong understanding of LLM inference systems, including batching, scheduling, KV-cache behavior, memory pressure, model parallelism, and GPU utilization.
- Experience building and operating GPU workloads on Kubernetes, including deployment, scaling, observability, and failure recovery.
- Strong hands-on systems engineering experience in architecture, implementation, profiling, and production debugging.
- Strong Python and a systems language such as Go, Rust, or C++.
- Understanding of Linux, containers, Kubernetes, networking, storage, and distributed-systems fundamentals.
- Experience with GPU observability, performance analysis, model formats, quantization, tokenization, and OpenAI-compatible serving APIs.
- Experience with traffic management, streaming, overload protection, health checking, failure recovery, identity, authorization, secrets, and workload isolation.
- Clear technical writing, design review, incident response, and cross-team communication skills.
- Experience contributing to KServe, vLLM, SGLang, or related inference, gateway, or orchestration projects is preferred.
- Experience modifying engine internals, profiling kernels, diagnosing CUDA or collective-communication issues, or working with mixture-of-experts models is preferred.
- Experience with data-sensitivity routing, customer-operated or disconnected environments, Kata Containers, confidential computing, or post-training rollouts is preferred.
Categories
About PhysicsX
PhysicsX builds an AI-driven simulation software stack that enables high-fidelity, multi-physics modeling and optimization for engineering and manufacturing teams. The company sells software and delivery services to enterprises in aerospace and defense, automotive, semiconductors, materials, and energy. It is privately held and headquartered in London, with offices in London and New York.