Staff Site Reliability Engineer
Hippocratic AI8 days ago
Gurgaon, IndiaStaff+
Responsibilities
- Design and build the GPU management and scheduling platform for approximately 30 models running across heterogeneous hardware.
- Create metrics pipelines and control logic for GPU utilization, admission control, request queuing or shedding, and autoscaling.
- Develop cloud orchestration systems and operators in Python and Go.
- Architect and operate scalable, fault-tolerant, secure production systems across AWS, GCP, or Azure.
- Build infrastructure automation and deployment pipelines using Terraform and CI/CD tools.
- Maintain monitoring, logging, and alerting systems for platform reliability and performance.
- Develop and enforce security and compliance policies for a healthcare AI platform.
- Partner with engineers and research scientists to resolve complex infrastructure, deployment, and operational issues.
- Mentor engineers and raise the team's technical bar.
Requirements
- 10+ years of professional experience across site reliability or DevOps engineering and software engineering.
- Required Computer Science degree from a top CS program.
- Strong software engineering fundamentals, including building orchestration and scheduling systems in Python and/or Go.
- Experience designing systems that interpret operational metrics and drive control loops such as autoscaling, load shedding, or admission control.
- Deep experience with infrastructure automation and CI/CD, including Terraform, GitLab CI/CD, or similar tools.
- Hands-on production experience with at least one major cloud platform: AWS, GCP, or Azure.
- Strong knowledge of Docker and Kubernetes.
- Experience with monitoring and logging stacks such as ELK, Grafana, Datadog, or similar tools.
- Familiarity with secrets management and security tooling such as HashiCorp Vault, AWS KMS, or Azure Key Vault.
- Strong problem-solving, independent working, collaboration, communication, and interpersonal skills.
- Experience managing GPU fleets or scheduling workloads across heterogeneous accelerators is preferred.
- Familiarity with ML inference serving and model deployment tools such as Triton, KServe, or Ray Serve is preferred.
- Experience with Kubernetes autoscaling internals, including HPA, VPA, custom metrics, and custom controllers, is preferred.
- Experience implementing HIPAA and SOC 2 compliance is preferred.
- Experience operating in an HPC environment is preferred.
- A bachelor's or master's degree in Computer Science, Computer Engineering, or a related field is preferred.
Benefits
- The role is based in the office five days a week in Gurugram.
- The position requires some flexibility for overlapping working hours with a US-based team in California.
Tech Stack
Categories
DevOpsSite Reliability
About Hippocratic AI
Hippocratic AI builds a safety-focused large language model and AI agents for healthcare workflows, used by health systems for patient outreach, post-discharge follow-up, and chronic-care management. It licenses its platform and tools to providers to automate and scale clinical support tasks while meeting health-system requirements. Founded in 2023 and headquartered in Palo Alto, the company is privately held.