10 months ago
Remote, United States +5 moreSenior
Responsibilities
- Own the reliability, performance, and observability of the entire inference stack.
- Design and refine telemetry pipelines for metrics, logs, and traces at large scale.
- Tune Kubernetes autoscalers to improve GPU efficiency.
- Create Terraform modules that embed resilience into new clusters.
- Harden request-routing and retry logic to handle transient failures.
- Build automation and runbooks to detect, isolate, and remediate incidents quickly.
- Drive post-mortems and improvements that prevent recurring incidents.
- Scale the platform while meeting aggressive cost and reliability targets.
Requirements
- Deep fluency with Kubernetes, Prometheus, Grafana, Terraform, and infrastructure-as-code.
- Comfortable scripting in Python or Bash.
- Understanding of alert design and service-level objectives for high-throughput APIs.
- Production experience with distributed back-end systems and their failure modes.
- Experience with GPU-heavy workloads using vLLM, Triton, Ray, or another accelerator stack is valuable.
- Background in MLOps or model-hosting platforms is valuable.
- Ability to debug performance from the kernel through the application layer and collaborate with software engineers.
Benefits
- Competitive salary and comprehensive benefits package.
- Opportunities for professional growth within Nebius.
- Flexible working arrangements.
- Dynamic and collaborative work environment that values initiative and innovation.
Tech Stack
Categories
DevOpsSite Reliability
About Nebius
Nebius builds a full-stack AI cloud offering GPU compute, storage, and tools for training and deploying ML models for startups, enterprises, and research labs. It sells consumption-based cloud infrastructure (IaaS/PaaS) and managed services tailored to generative AI workloads, including large-scale model training and inference. The company is headquartered in Amsterdam and operates as an independent provider.