Senior AI DevOps / LLMOps
TechBiz Global GmbH3 months ago
Remote, WorldwideSenior
Responsibilities
- Design and implement AI-focused CI/CD pipelines for model weights, dataset versioning, application code, prompts, and agentic workflows.
- Provision and manage GPU clusters, TPU pods, and hybrid on-premises/cloud environments using Terraform, Pulumi, or Ansible.
- Define policy-as-code for AI endpoints covering security, cost limits, and data residency.
- Architect progressive delivery strategies including canary, blue-green, and shadow deployments for AI systems.
- Build evaluation gates and A/B testing frameworks for LLM outputs and agentic behavior, including bias, hallucination, and performance checks.
- Establish inference observability for token throughput, latency, accuracy drift, and production edge cases.
- Create feedback loops that inform model training and fine-tuning pipelines.
Requirements
- 10+ years of experience in DevOps, SRE, or cloud engineering.
- 2+ years of hands-on MLOps or LLMOps experience moving LLMs from notebooks to production.
- Proven experience managing hybrid cloud environments such as AWS or Azure combined with a private data center.
- Advanced Kubernetes skills, specifically with KubeFlow, Ray, or NVIDIA Triton.
- Expertise with GitHub Actions or GitLab CI and Terraform or Pulumi.
- Experience with AI tooling such as Weights & Biases, MLflow, LangSmith, or Arize Phoenix.
- Understanding of GPU virtualization, CUDA drivers, and on-premises hardware management.
- Familiarity with Open Policy Agent and Vault for policy and secret management.
Tech Stack
Categories