6 months ago
San José, Costa Rica +2 moreSenior
Responsibilities
- Own and operate GPU and accelerator clusters for model training, inference, evaluation, and experimentation.
- Design local GPU infrastructure and compute systems that reduce external-provider dependency and control costs.
- Build scheduling, orchestration, placement, quota management, workload isolation, and utilization systems across heterogeneous accelerators.
- Optimize inference pipelines for latency, throughput, reliability, memory efficiency, and cost using serving frameworks such as vLLM, Triton Inference Server, and TensorRT.
- Partner with ML engineers and researchers to improve training, evaluation, batch inference, online inference, deployment, and production-debugging workflows.
- Build observability for GPU utilization, memory pressure, queue depth, saturation, token throughput, request latency, failed workloads, capacity pressure, and spend.
- Drive reliability engineering, incident response, alerting, runbooks, and post-incident improvements for always-on AI compute infrastructure.
- Evaluate and integrate new hardware, cloud instance families, specialized accelerators, runtimes, schedulers, and serving frameworks.
- Build internal tooling that makes GPU usage visible, accountable, and easier for teams to consume.
- Contribute to architecture decisions balancing performance, cost efficiency, scalability, operational simplicity, and production safety.
Requirements
- 5+ years of infrastructure engineering experience, including significant experience with GPU compute, ML infrastructure, distributed systems, high-performance computing, or large-scale production platforms.
- Hands-on experience operating GPU clusters or accelerator-backed infrastructure in production or production-like environments, including scheduling, orchestration, utilization monitoring, and cost optimization.
- Strong systems engineering fundamentals across Linux, networking, storage, containers, Kubernetes, distributed runtimes, and production debugging.
- Experience with ML serving frameworks such as vLLM, Triton Inference Server, TensorRT, TorchServe, KServe, Ray Serve, or equivalent systems.
- Proficiency in Python for infrastructure automation, tooling, debugging, integration, and operational workflows.
- Understanding of performance tradeoffs involving batching, concurrency, memory usage, GPU utilization, model size, latency, throughput, availability, and cost.
- Track record of optimizing compute costs while maintaining performance, reliability, and availability expectations.
- Experience building observable systems with metrics, logs, traces, dashboards, alerts, and incident workflows.
- Ability to work in high-stakes, always-on environments where uptime, throughput, correctness, and operational discipline are critical.
- Clear communication skills for translating infrastructure tradeoffs to researchers, product teams, platform engineers, security stakeholders, and engineering leadership.
- Preferred experience includes work at a frontier AI lab, hyperscaler, high-frequency trading firm, research platform, or high-scale ML organization.
- Preferred qualifications include familiarity with custom silicon or specialized accelerators, capacity planning, procurement input, reserved capacity strategy, cloud accelerator economics, or GPU fleet cost management.
- Preferred experience includes distributed training frameworks such as DeepSpeed, Megatron-LM, FSDP, or Ray; debugging CUDA, NCCL, kernels, drivers, runtimes, memory, or networking; and using Rust, C++, Go, or CUDA.
- Crypto, financial services, trading infrastructure, or security-sensitive production infrastructure experience is a plus.
Categories
About Kraken
Kraken builds a global digital asset trading platform for individuals and institutions, offering spot, margin, futures, staking, OTC, and related infrastructure. The company generates revenue primarily from trading and financing fees and institutional services. Founded in 2011 and operated by privately held Payward, it is remote-first and serves customers worldwide; it has published cryptographic Proof of Reserves since 2014 to verify client asset holdings.