Responsibilities
- Own production reliability for high-SLA and complex customer environments.
- Define and evolve per-tenant SLOs and reliability models while reducing SLO budget burn.
- Design and implement automation, self-healing, auto-scaling, and other solutions to eliminate toil and improve reliability.
- Serve as a primary escalation point and participate in on-call and customer-impacting incident response.
- Lead incident investigations, post-incident reviews, and customer communications through bridge calls when necessary.
- Improve monitoring, alert quality, customer observability, and noisy escalation handling.
- Design fault-tolerant and scalable systems across the service lifecycle.
- Partner with Mimir, Loki, and Tempo product engineering squads on feature design, technical designs, roadmaps, and product strategy.
- Contribute to design documents, code reviews, and pull request reviews.
- Teach Site Reliability Engineering practices and mentor engineers.
Requirements
- At least 8 years of engineering experience, including at least 4 years in SRE, customer reliability engineering, or production engineering.
- Strong Kubernetes experience in AWS, GCP, or Azure and familiarity with Helm, Terraform, or Jsonnet.
- Experience operating multi-tenant production systems and designing and implementing SLOs.
- Experience with one or more programming languages such as Go, Python, or Java.
- Experience with Linux operating system internals and knowledge of networking, cloud storage, and scaling.
- Strong technical leadership experience, including leading projects, mentoring engineers, and serving as a force multiplier.
- Experience participating in incident response, conducting investigations, following up on actions, and writing high-quality post-incident reviews.
- Ability to reason about performance, scaling, failure modes, and production reliability.
- Ability to partner deeply with product engineering teams and work autonomously.
- Formal customer reliability engineering experience is strongly preferred.
Benefits
- 100% remote work opportunity for candidates in the UK, Sweden, Spain, or Germany.
- In-person onboarding is provided.
- 30 days of annual leave per year, including 3 Grafana Shutdown Days, subject to local legislation.
- Equity and bonus may be available.
- Career growth pathways, transparent communication, and an autonomy-focused global culture.
Categories
About Grafana
Grafana Labs, the company behind the open observability cloud, is founded on the principles of open source, open standards, open ecosystems, and open culture. Grafana Cloud, our fully managed observability platform, is flexible and built for scale. With Grafana Cloud's actually useful AI, organizations can see, understand, and act on all their disparate data to move at the speed of their ambitions, while getting the visibility they need to run AI systems reliably and at scale. Today, more than 35 million users and 7,000+ customers – including Anthropic, Bloomberg, NVIDIA, Microsoft, and Salesforce – trust Grafana Labs to ensure reliability of their applications and systems, resolve incidents quickly, and optimize their telemetry to reduce noise and cost. We are a 100% remote company with 1,400+ team members across 40+ countries, and we’re backed by leading investors including Lightspeed Venture Partners, Sequoia Capital, GIC, Coatue, J.P. Morgan, CapitalG, and Lead Edge Capital.