OpenAI

Software Engineer, Hardware Health

OpenAI
Apply
4 months ago

Base Salary

$250k - $445k/yr

Responsibilities

  • Define and maintain health signals across GPUs, CPUs, networking, and platform infrastructure.
  • Build health checks that detect, remediate, and verify failures at scale with minimal latency.
  • Investigate hardware failures and system-level issues across large-scale compute environments.
  • Own node lifecycle workflows including drain, quarantine, repair, RMA, and return to service.
  • Build automation and tooling for global cluster management with minimal manual intervention.
  • Partner with workload, reliability, and provider teams to integrate health signals into training and inference systems.

Requirements

  • 7+ years of industry experience in software or infrastructure engineering.
  • Strong proficiency with Python and shell scripting.
  • Experience building large-scale distributed systems or infrastructure platforms.
  • Comfort analyzing noisy operational data with SQL, PromQL, or similar tooling.
  • Experience building reproducible analyses and operational tooling.
  • Strong systems debugging and operational instincts with an ownership mindset.
  • Bonus experience includes low-level hardware systems, Linux tooling, large-scale GPU or accelerator clusters, network operations, observability, automated remediation, fleet lifecycle management, and improving reliability or workload uptime.

Tech Stack

Categories

OpenAI

About OpenAI

1,001-5,000 employees

OpenAI is an AI research and deployment company dedicated to ensuring that general-purpose artificial intelligence benefits all of humanity. AI is an extremely powerful tool that must be created with safety and human needs at its core. OpenAI is dedicated to putting that alignment of interests first — ahead of profit. To achieve our mission, we must encompass and value the many different perspectives, voices, and experiences that form the full spectrum of humanity. Our investment in diversity, equity, and inclusion is ongoing, executed through a wide range of initiatives, and championed and supported by leadership. At OpenAI, we believe artificial intelligence has the potential to help people solve immense global challenges, and we want the upside of AI to be widely shared. Join us in shaping the future of technology.

Contact me