DevOps Engineer - AI Runtime Services
Demandbaseabout 3 hours ago
Hyderābād, IndiaMid Level / Senior
H1B Sponsor
Responsibilities
- Manage the LLM gateway, ensuring uptime and performance.
- Oversee reliability through SLOs, incident response, and postmortems.
- Implement cost control measures for AI model usage.
- Enhance LLM observability with metrics and logging.
- Develop evaluation and experimentation infrastructure for AI features.
- Implement caching and guardrails for safety and efficiency.
Requirements
- Strong production infrastructure experience with Kubernetes, AWS, Terraform, and GitOps.
- Proficient in Python for developing services and tooling.
- Experience with on-call duties and incident response.
- Fluency in observability practices and metrics instrumentation.
- Hands-on familiarity with LLM systems and their operational aspects.
Benefits
- Comprehensive medical, personal accident, and term life insurance.
- Preventive healthcare coverage including dental, vision, and OPD.
- Strong mental health support and fitness benefits.
- Car lease policy and gratuity for long-term financial well-being.