Principal SRE Engineer
Eli Lilly and Company24 hours ago
Hyderābād, IndiaStaff+
Responsibilities
- Design and build self-healing automation, including circuit breakers, graceful degradation, and automated remediation.
- Run resilience and chaos testing to validate automation before production use.
- Author, validate, maintain, and retire remediation runbooks covering execution order, rollback steps, and exception handling.
- Lead or contribute to root-cause analysis, blameless postmortems, incident response, and durable engineering fixes.
- Track remediation effectiveness through recurrence reduction and escalate fixes requiring broader engineering investment.
- Partner with agentic automation engineering and Operations teams on safe remediation, confidence thresholds, human-in-the-loop boundaries, and autonomy graduation criteria.
- Act as incident commander for escalations within the role’s area and ensure compliance with change-control, audit, and validated-environment standards.
- Mentor reliability and automation engineers on runbook quality and self-healing design.
Requirements
- 10+ years of progressive engineering experience, including meaningful experience as a Site Reliability Engineer, Production Engineer, or equivalent.
- Hands-on ownership of self-healing automation or runbook-driven remediation for a multi-application production estate.
- Production reliability experience in a regulated or audited environment such as GxP, SOX, HIPAA, or PCI.
- Experience authoring and validating runbooks with safe execution order, rollback steps, and exception handling.
- Hands-on fluency with observability tools, Terraform, CI/CD pipeline hardening, Kubernetes, container platforms, and at least one major public cloud.
- Experience designing self-healing patterns and validating them before production deployment.
- Experience contributing to root-cause analysis and blameless postmortems and converting findings into durable fixes.
- Bachelor’s degree or higher in Computer Science, Information Technology, or a closely related engineering field.
- Preferred qualifications include chaos engineering or resilience-testing experience, AWS reliability-service expertise, AIOps or agent-assisted operations experience, autonomy-criteria experience, runbook-library scaling experience, and experience in regulated industries.
Benefits
- Onsite role based in Hyderabad.
- Flexible work hours and different shifts may be required to support global delivery partners and continuous operations.
- Some weekend and holiday work may be required, with appropriate benefits adjustments where applicable.
- Lilly provides disability accommodations and is committed to an inclusive, nondiscriminatory workplace.
Tech Stack
Categories
Site Reliability
About Eli Lilly and Company
Eli Lilly and Company researches, develops, manufactures, and markets prescription medicines and biologics for patients, sold through healthcare providers, pharmacies, and payers. Core therapeutic areas include diabetes, oncology, immunology, neuroscience, and obesity. Founded in 1876 and headquartered in Indianapolis, it is a public company (NYSE: LLY) with global operations and is building a new advanced manufacturing site for gene therapy and other modalities in Lebanon, Indiana.