5 months ago
Remote, United Kingdom +3 moreStaff+
Responsibilities
- Write production software and reliability improvements in services, tooling, and the Mimir, Loki, and Tempo codebases.
- Build automation and platform tooling across regions and cloud providers to reduce operational toil.
- Define and evolve per-tenant SLOs and reliability models, using incident learnings to improve systems.
- Lead technical design, review designs and pull requests, and ensure systems are scalable and operable.
- Improve observability and alerting to detect problems early and make escalations actionable.
- Participate in weekday and weekend on-call rotations, lead customer-impacting incident response, and write post-incident reviews.
- Investigate customer escalations, tune capacity and configuration, and convert recurring issues into permanent fixes.
- Drive cross-team technical direction, mentor engineers, and contribute to roadmaps and product strategy.
Requirements
- 8+ years of software engineering experience, including significant experience building and owning distributed systems in production.
- Strong production software development skills, ideally in at least one object-oriented language, with the ability to modify large open-source codebases.
- Deep understanding of distributed-systems performance, scaling, and failure modes.
- Experience operating multi-tenant systems at scale on Kubernetes in AWS, GCP, or Azure.
- Experience with infrastructure-as-code tools such as Terraform, Helm, or Jsonnet.
- Hands-on experience designing and implementing SLOs and participating in production on-call and blameless incident response.
- Staff-level technical leadership, including cross-team project leadership, technical direction, and mentoring.
- Working knowledge of Linux internals, networking, and cloud storage.
- Experience with Mimir, Loki, Tempo, Pyroscope, Prometheus, or other observability and time-series databases is preferred.
- Open-source contributions and customer reliability engineering or customer-facing escalation experience are preferred.
Benefits
- 100% remote work for candidates in the UK, Sweden, Spain, or Germany.
- In-person onboarding.
- 30 days of annual leave per year, including three Grafana Shutdown Days, subject to local legislation.
- Equity, bonus where applicable, and other country-specific benefits.
- Global, transparent, high-trust culture with career growth opportunities.
Tech Stack
Categories
BackendSite Reliability
About Grafana
Grafana Labs builds open-source observability tools and a managed SaaS, Grafana Cloud, used by engineering and SRE teams to monitor, visualize, and analyze telemetry. Its products include Grafana dashboards plus Loki (logs), Tempo (traces), and Mimir (metrics), offered as cloud subscriptions and enterprise software. Privately held and headquartered in New York, it serves customers such as Anthropic, Bloomberg, NVIDIA, Microsoft, and Salesforce.