4 days ago
Hyderābād, IndiaMid Level / Senior
Responsibilities
- Operate and optimize multi-tenant LLM inference, model serving, VectorDB/RAG, and MCP platforms across cloud and GPU clusters.
- Establish SLOs, SLIs, health checks, and observability dashboards using Prometheus, Grafana, Splunk, and OpenTelemetry.
- Build automation for rollouts, autoscaling, canary deployments, self-healing, and disaster recovery.
- Tune GPU scheduling, inference throughput, latency, batching, caching, and resource allocation.
- Lead complex production incidents, perform root-cause analyses, and implement systemic remediations.
- Partner with researchers, product developers, and platform teams on production readiness reviews and participate in an on-call rotation.
Requirements
- Bachelor’s degree in computer science or engineering with 5+ years of relevant experience, or a master’s degree with 3+ years of relevant experience.
- At least 3 years of programming experience in Python, Go, or Java building automation, tooling, or production backend services.
- Experience running distributed, containerized production systems using Kubernetes, microservices, and CI/CD.
- Experience with production operations, distributed troubleshooting, and monitoring stacks such as Prometheus, Grafana, Splunk, or OpenTelemetry.
- Hands-on working knowledge of AWS, Azure, or GCP.
- Preferred experience with vLLM, TensorRT-LLM, Ray Serve, KServe, GPU scheduling, capacity planning, model parallelism, and performance profiling.
- Preferred experience operating vector databases, search engines, or distributed cache layers such as Milvus, Pinecone, or Redis.
- Familiarity with RAG pipelines, model routing, Model Context Protocol, agent runtimes, multi-region high availability, automated remediation, and chaos engineering.
Tech Stack
Categories
About Cisco
Cisco designs and sells networking, security, and collaboration platforms for enterprises, service providers, and governments, spanning routers and switches, Wi‑Fi, firewalls, zero‑trust, observability, and cloud-managed IT (Meraki) plus Webex. Its business model mixes hardware, software subscriptions, and support/consulting services. Founded in 1984 and headquartered in San Jose, California, Cisco is a public company traded on Nasdaq and serves customers across data centers, campuses, and service provider networks.