Staff AI Scheduling & Orchestration Engineer

1 month ago

Singapore, SingaporeStaff+

Responsibilities

Design and implement advanced batch scheduling architectures using Volcano or YuniKorn for multi-node gang scheduling.
Develop cluster-wide admission control and job queueing mechanisms using Kueue for high-volume AI workload traffic.
Use Kubernetes Dynamic Resource Allocation and custom scheduler plugins to manage accelerator requests.
Architect topology-aware pod placement optimized for NVLink and InfiniBand communication.
Implement GPU sharing technologies such as MIG and time-slicing, along with multi-tenancy isolation policies.
Integrate scheduling systems with bare-metal hardware and storage I/O patterns in collaboration with GPU Systems and Storage teams.
Improve reliability and scalability while resolving resource contention and deadlocks in large-scale HPC environments.
Mentor junior engineers and conduct design reviews.

Bachelor’s or master’s degree in Computer Science, Electrical Engineering, or a related field.
At least 6 years of distributed systems engineering experience with hands-on Kubernetes scheduling frameworks and orchestrators.
Extensive experience with AI workload execution patterns and distributed training frameworks such as PyTorch Distributed, Ray, and MPI.
Experience operating, debugging, and scaling scheduling stacks in HPC or large-scale production cloud environments.
Strong knowledge of GPU hardware architectures and scheduling challenges for distributed AI training and inference.
Experience with infrastructure automation and infrastructure-as-code, such as Terraform and Go-based Operators.
Strong technical communication and leadership skills, with the ability to influence cross-functional teams and align architectural goals.
Ability to translate complex, ambiguous requirements into concrete, scalable engineering solutions in a high-velocity environment.

Inclusive and respectful workplace with an emphasis on authenticity and diverse perspectives.
Opportunity to work on new projects and contribute directly to the digital asset and AI cloud industries.
Networking opportunities with industry pioneers and enthusiasts.
Personal accountability, autonomy, fast growth, and learning opportunities.
Welfare benefits and developmental opportunities including training and mentoring.

201-500 employees