Responsibilities
- Implement, maintain, and optimize AI platform capabilities deployed on Google Kubernetes Engine and Google Cloud Platform.
- Deploy, configure, and scale self-hosted large language models and production workloads on Kubernetes.
- Build platform components, custom Kubernetes resources, automation, operators, and telemetry pipelines using Python or Go.
- Apply software engineering practices including code quality, automated testing, CI/CD, GitOps, documentation, and reusable platform templates.
- Operate and troubleshoot cloud-native networking, service mesh, GPU resource management, observability, and model-serving infrastructure.
- Collaborate with application, data science, and platform teams to troubleshoot integration issues and improve developer experience.
- Execute technical roadmaps that improve platform reliability, reduce operational overhead, and accelerate AI product delivery.
Requirements
- 5+ years of experience in platform or software engineering.
- Hands-on experience deploying, configuring, and scaling self-hosted LLMs on GKE using vLLM, NVIDIA NIM, or SGLang.
- Understanding of Kubernetes GPU allocation, multi-GPU nodes, and model execution requirements.
- Experience with cloud-native AI infrastructure or control planes such as Agentgateway or Kagent.
- Practical experience configuring OpenTelemetry instrumentation and collectors, including GenAI semantic conventions for traces, metrics, token usage, and latency.
- Experience integrating GenAI frameworks or services such as LangGraph, LangChain, Google Agent Development Kit, Google Vertex AI, Google Agentspace, or Gemini APIs.
- Production experience deploying and supporting Kubernetes workload pipelines with attention to latency, availability, and resource utilization.
- Practical experience with GKE, Google Compute Engine, GCP infrastructure, Istio Ambient Mode or sidecar-based Istio, container runtimes, Kubernetes GPU device plugins or operators, OpenTelemetry Collectors, GitOps, and Terraform.
- Strong proficiency in Python or Go for automation, custom tooling, scripts, or operators.
- Understanding of design patterns, unit and integration testing, clean code principles, maintainable software, documentation, and cross-team technical communication.
Tech Stack
Categories
About CME Group
As the world's leading derivatives marketplace, CME Group (www.cmegroup.com) is where the world comes to manage risk. CME Group exchanges offer the widest range of global benchmark products across all major asset classes, including futures and options based on interest rates, equity indexes, foreign exchange, energy, agricultural commodities, metals, weather and real estate. CME Group brings buyers and sellers together through its CME Globex® electronic trading platform and its trading facilities in New York and Chicago. CME Group also operates CME Clearing, one of the world’s leading central counterparty clearing provider in the world, which offers clearing and settlement services for exchange-traded contracts, as well as for over-the-counter derivatives transactions through CME ClearPort®. These products and services ensure that businesses everywhere can substantially mitigate counterparty credit risk in both listed and over-the-counter derivatives markets.