Senior Production Engineer
Clear Street17 hours ago
London, United KingdomSenior
Responsibilities
- Design and build monitoring and observability platforms that reduce alert fatigue and accelerate root-cause analysis.
- Develop intelligent automation and self-healing capabilities that trigger recovery workflows and reduce mean time to recovery.
- Analyze incidents and engineer solutions that prevent recurring classes of failures.
- Build reusable runbooks, diagnostic tooling, and recovery playbooks.
- Create reliable golden-path operational workflows for engineering teams.
- Partner with Platform Engineering on CI/CD pipelines, deployment safety, and infrastructure resilience.
- Champion Infrastructure as Code, GitOps, and SRE practices.
- Measure production health through SLIs, SLOs, and SLAs and use reliability data to guide engineering priorities.
- Build AI-assisted diagnostics and developer tooling where appropriate.
- Reduce mean time to detection and recovery, operational toil, and manual incident intervention.
Requirements
- Strong hands-on Python skills for automation and tooling.
- Experience in SRE, Production Engineering, Platform Engineering, or a related discipline with direct production ownership.
- A proven record of building automation and diagnostic tooling that improves recovery times or reduces operational toil.
- Deep familiarity with Kubernetes, containers, distributed systems, and production failure modes.
- Experience with observability platforms such as Datadog.
- Exposure to Terraform and GitOps-based deployment workflows using ArgoCD, GitHub Actions, or similar tools.
- Familiarity with Java, Go, Kafka, Redis, Snowflake, and Postgres.
- Strong analytical, problem-solving, communication, and cross-functional collaboration skills.
- A product mindset for operational tooling, including usability, adoption, and documentation.
- Self-directed initiative, curiosity, and continuous learning; fintech or financial industry experience is a plus.
Tech Stack
Categories
Site Reliability
About Clear Street
Clear Street builds a cloud-native brokerage and clearing platform for institutional investors, combining prime brokerage, clearing, and custody with APIs and modern infrastructure. It serves hedge funds, professional traders, and other market participants, helping them access multiple asset classes and markets through a unified system. Founded in 2018 and headquartered in New York, the company is privately held and operates regulated entities supporting global capital markets.