5 months ago
Base Salary
$196k - $230k/yr
Responsibilities
- Serve as a senior technical leader for Robinhood’s reliability and observability strategy.
- Coordinate production incident mitigation, including service owners, rollbacks, traffic shifts, and incident communications.
- Develop and maintain incident management processes, procedures, education, adoption, and measurement.
- Own company-wide incident discovery through dashboards and alerts tied to critical user journeys, availability, and business-impact metrics.
- Build and evolve incident response tooling and frameworks for monitoring, alerting, and observability across hundreds of services and systems.
- Lead post-incident governance, including postmortems, SEV reviews, and follow-up tracking.
- Design failure-mitigation strategies that avoid full-region or full-datacenter failovers.
- Define the observability roadmap for Robinhood’s critical user journeys.
- Deliver executive-level reporting on service quality and reliability.
- Mentor engineers and contribute to hiring and engineering culture.
Requirements
- At least 5 years of software engineering experience, including significant experience operating production systems.
- At least 2 years focused on reliability engineering, infrastructure, distributed systems, or production operations.
- Hands-on experience serving as an incident leader, incident commander, IMOC, or primary on-call.
- Deep knowledge of systems reliability, observability frameworks, and fault-tolerant architecture design.
- Experience with multi-region or multi-cluster architectures, capacity planning, and failover strategies.
- Familiarity with modern observability stacks such as OpenTelemetry, Prometheus, and Grafana.
- Demonstrated ability to improve MTTD, MTTR, availability, or customer impact measurably.
- Strong communication and cross-functional collaboration skills during high-severity incidents.
Benefits
- In-person work in the Menlo Park, California office is expected at least 3 days per week.
- Performance-driven compensation includes bonus programs, equity ownership, and 401(k) matching.
- Benefits include paid health insurance, dependent coverage, lifestyle wallet, life and disability insurance, fertility benefits, mental health benefits, paid time off, holidays, sick time, and parental leave.
- Office experience includes catered meals, events, and comfortable workspaces.
Tech Stack
GrafanaPrometheus
Categories
DevOpsSite Reliability
About Robinhood
Trade. Invest. Earn. rbnhd.co/social_media_disclosures
