5 months ago
Base Salary
$250k - $445k/yr
Responsibilities
- Define and maintain health signals across GPUs, CPUs, networking, and platform infrastructure.
- Build health checks that detect, remediate, and verify failures at scale with minimal latency.
- Investigate hardware failures and system-level issues across large-scale compute environments.
- Own node lifecycle workflows including drain, quarantine, repair, RMA, and return to service.
- Build automation and tooling for global cluster management with minimal manual intervention.
- Partner with workload, reliability, and provider teams to integrate health signals into training and inference systems.
Requirements
- 7+ years of industry experience in software or infrastructure engineering.
- Strong proficiency with Python and shell scripting.
- Experience building large-scale distributed systems or infrastructure platforms.
- Comfort analyzing noisy operational data with SQL, PromQL, or similar tooling.
- Experience building reproducible analyses and operational tooling.
- Strong systems debugging and operational instincts with an ownership mindset.
- Bonus experience includes low-level hardware systems, Linux tooling, large-scale GPU or accelerator clusters, network operations, observability, automated remediation, fleet lifecycle management, and improving reliability or workload uptime.
About OpenAI
OpenAI builds and deploys large-scale AI models and tools—including ChatGPT, GPT-4–class models, DALL·E, and Whisper—sold via APIs and enterprise subscriptions to developers and businesses. It monetizes through usage-based API pricing and ChatGPT Plus/Team/Enterprise, and also reaches customers via Microsoft’s Azure OpenAI Service. Founded in 2015 and headquartered in San Francisco, it operates as a private partnership.