Senior Systems Reliability Engineer
Electric Reliability Council of Texas (ERCOT)3 months ago
Taylor, TX, USASenior
Base Salary
$109k - $150k/yr
Responsibilities
- Build production reliability software, automated remediation systems, self-healing infrastructure components, and operational automation frameworks.
- Own SLO and error budget governance, SLI instrumentation, alert quality, observability architecture, and production readiness standards.
- Lead high-severity incident response, dual-datacenter failover execution, root-cause analysis, post-mortems, and remediation tracking.
- Design and implement chaos engineering programs, resilience tests, CI/CD reliability gates, canary analysis, progressive delivery validation, and rollback triggers.
- Diagnose Java/Spring Boot and JVM performance issues involving heap pressure, garbage collection, thread pools, connection leaks, and class loading.
- Design and operate Kubernetes and OpenShift workloads and build infrastructure-as-code using Terraform, Ansible/AAP, and Azure Resource Manager templates.
- Implement NERC/CIP compliance controls, configuration drift detection, access validation, audit evidence, and regulatory audit preparation.
- Mentor Systems Reliability Specialist I and II employees, lead design reviews, establish reliability engineering standards, and contribute shared tooling and technical documentation.
- Participate in a 24/7 on-call rotation and coordinate team delivery and on-call activities as needed.
Requirements
- Minimum 5 years of progressive experience in systems reliability, software with an SRE focus, or a closely related discipline.
- Bachelor's degree in Computer Science, Software Engineering, MIS, or a related field; equivalent education and experience providing comparable knowledge is accepted.
- Production experience building and operating SLO frameworks, observability platforms, chaos engineering programs, and automated remediation tooling.
- Proficiency in Python and Java, with experience writing production-quality reliability tooling and automation; Bash and Linux experience required.
- Deep Java, Spring Boot, and JVM performance expertise, including heap analysis, garbage collection tuning, thread profiling, and application instrumentation.
- Experience with Kubernetes, OpenShift, CI pipelines, Open telemetry, infrastructure-as-code, and high-severity incident response.
- Expertise with Grafana LGTM tooling, including Loki, Grafana, Tempo, and Mimir, plus Dynatrace and Splunk.
- Experience with Terraform, Ansible/AAP, or equivalent; networking knowledge and experience with container networking troubleshooting.
- Experience leading blameless post-mortem programs; dual-datacenter or hybrid-cloud reliability architecture experience preferred.
- NERC/CIP compliance experience involving control implementation, audit preparation, and regulatory engagement preferred.
- Master's degree in Computer Science, Software Engineering, or a related field is preferred.
- Azure certification, Certified Kubernetes Administrator certification, and ITIL Foundation or Managing Professional certification are preferred.
Benefits
- Hybrid work location in Taylor, Texas, with two days per week onsite.
- Expected salary range of $109,000 to $150,000 per year.
- 24/7 on-call rotation participation is part of the role.
Tech Stack
AnsibleApache KafkaAzureBashDatadogGrafanaJavaKubernetesLinuxOpenShiftPostgreSQLPythonSOAPSplunkSpring BootTerraform
Categories
Site Reliability
About Electric Reliability Council of Texas (ERCOT)
The Electric Reliability Council of Texas (ERCOT) is the independent system operator for most of Texas, operating the state’s electric grid and competitive power markets as a membership-based 501(c)(4) nonprofit. It schedules generation, manages transmission, settles the wholesale market, and administers retail switching. ERCOT serves about 27 million customers—roughly 90% of Texas’s load—over a grid with 54,100+ miles of transmission and 1,250+ generation units. Headquarters: Taylor, Texas.