2 days ago
Base Salary
$160k - $300k/yr
Responsibilities
- Design, deploy, and operate GPU clusters across AWS, GCP, other hyperscalers or neoclouds, and on-premises data centers.
- Build Kubernetes and Slurm operators, metadata services, accelerator schedulers, quotas, fair-share systems, and bin-packing capabilities.
- Run distributed training and high-throughput LLM serving infrastructure with checkpointing, model loading, parallelism, quantization, and KV-cache optimization.
- Develop benchmarking, traffic replay, health-check, automated remediation, fault-diagnosis, and fleet reliability systems.
- Instrument utilization and reliability with observability tools, recording rules, and alerts.
- Ingest utilization, occupancy, and billing data; support cost attribution, capacity planning, autoscaling, rightsizing, and idle-capacity recovery.
- Build internal tools, dashboards, runbooks, and documentation for researchers, finance, leadership, and other users.
- Own infrastructure projects end to end and participate in on-call for critical systems.
Requirements
- At least 5 years of experience building and operating production distributed systems or ML infrastructure; 7 or more years are specified for Staff-level scope.
- Production-quality Python and proficiency in either Go or C++, plus shell scripting for system administration.
- Deep hands-on Kubernetes and container experience, with working knowledge of Slurm or another HPC resource manager.
- Experience operating GPU workloads at scale, including scheduling, orchestration, and troubleshooting.
- Hands-on experience with at least one distributed training or serving stack, such as PyTorch Distributed, Ray, vLLM, or Megatron-LM.
- Deep experience with at least one major cloud provider.
- Experience with observability tools such as DataDog, Prometheus, PromQL, and Grafana, and comfort supporting critical systems on call.
- Familiarity with or willingness to learn LLM inference optimization, high-performance networking, GPU profiling, CUDA or Triton kernels, non-NVIDIA accelerators, capacity planning, cost attribution, automated fault diagnosis, configuration management, and secure or air-gapped deployments.
- Ability to own projects end to end and gather requirements with little direction.
Benefits
- Bi-annual performance bonus structure and generous equity grant.
- Relocation, housing, meal, laundry, and personal wellness support.
- Free Equinox membership.
- Health, dental, and vision insurance.
- In-person work five days per week in the San Francisco, New York City, or London office.
Tech Stack
Categories
About Mercor
Mercor builds an expert-powered platform that trains, evaluates, and deploys AI systems for AI labs and enterprises. It operates APEX to assess model performance on economically valuable tasks, and provides services such as custom AI agents and teams of vetted domain experts who encode organizational knowledge into AI. Founded in 2023 and headquartered in San Francisco, it is privately held and works with frontier AI labs and enterprise clients requiring strict data isolation.