3 months ago
Base Salary
$196k - $230k/yr
Responsibilities
- Lead incident mitigation by coordinating service owners, facilitating rollbacks and traffic shifts, and maintaining a clear source of truth during active incidents.
- Develop and maintain incident management processes, procedures, education, adoption, and measurement programs.
- Define company-wide dashboards and alerts for critical user journeys, availability, and business-impact metrics.
- Own and evolve incident response tooling and measure improvements in MTTD and MTTR.
- Drive post-incident governance, including postmortems, SEV reviews, and follow-up tracking.
- Design failure-mitigation strategies that avoid full-region or full-datacenter failovers.
- Build monitoring, alerting, and observability frameworks across hundreds of services and systems.
- Own the roadmap for observability across Robinhood’s critical user journeys.
- Deliver executive-level reporting and insights on service quality and reliability.
- Mentor engineers, provide technical influence, and contribute to hiring and engineering culture.
Requirements
- At least 5 years of software engineering experience, including significant experience operating production systems.
- At least 2 years focused on reliability engineering, infrastructure, distributed systems, or production operations.
- Hands-on experience serving as an incident commander, IMOC, primary on-call, or in a comparable incident leadership role.
- Deep knowledge of systems reliability, observability frameworks, and fault-tolerant architecture design.
- Experience with multi-region or multi-cluster architectures, capacity planning, and failover strategies.
- Familiarity with modern observability stacks such as OpenTelemetry, Prometheus, and Grafana.
- Demonstrated ability to improve MTTD, MTTR, availability, or customer impact measurably.
- Strong communication and cross-functional collaboration skills, particularly during high-severity incidents.
Benefits
- Performance-driven compensation with bonus programs, equity ownership, and 401(k) matching.
- 100% paid employee health insurance with 90% dependent coverage, plus life and disability insurance.
- Lifestyle wallet for wellness, learning, and other benefits spending.
- Fertility benefits, mental health benefits, parental leave, paid time off, sick time, and company holidays.
- Catered meals, events, and comfortable workspaces.
- Based in New York, New York, with in-person attendance expected at least 3 days per week.
Tech Stack
GrafanaPrometheus
Categories
DevOpsSite Reliability
About Robinhood
Trade. Invest. Earn. rbnhd.co/social_media_disclosures
