3 days ago
Responsibilities
- Participate in on-call rotations, incident response, and blameless postmortems.
- Manage node lifecycles using infrastructure as code, including OS images, firmware, configuration management, kernels, and drivers.
- Build cluster, node, storage, and job/workflow observability for administrators and end users.
- Reduce operational toil through automation while operating production systems and writing supporting software.
- Manage compute and storage resources and lead capacity planning with users across the company.
- Collaborate with HPC systems engineers and engineers across disciplines to deliver operable and maintainable infrastructure.
Requirements
- Bachelor’s degree in computer science, engineering, mathematics, or a scientific discipline, or 2+ years of professional experience operating production infrastructure in lieu of a degree.
- 8+ years of experience with Linux operating systems in production.
- 8+ years of experience operating production infrastructure, including monitoring, debugging, and repairing owned systems.
- Preferred experience with SRE, DevOps, or production infrastructure engineering; monitoring and alerting; configuration management or infrastructure as code; scripting; containers; Kubernetes administration; and distributed or high-performance storage.
- Familiarity with HPC clusters, schedulers, GPU compute, scientific computing, ML training, or AI inference workloads is preferred but not required.
- Ability to communicate clearly with users, peers, and vendors and work with mission-critical and sensitive systems.
- Eligibility for access to classified material up to TS/SCI with polygraph.
- Must be able to participate in on-call rotations and work extended hours and weekends as needed for incidents, cluster bring-up, and time-critical failures.
- Must satisfy ITAR eligibility requirements or be eligible to obtain required U.S. Department of State authorizations.
Categories
Site Reliability
About SpaceX
SpaceX designs, manufactures, and launches orbital rockets and spacecraft, and operates Starlink, a global satellite internet network for consumers, businesses, and governments. Its revenue comes from commercial and government launch services (Falcon 9/Falcon Heavy, Dragon cargo and crew to the ISS) and subscription broadband with Starlink hardware and service. Founded in 2002 and headquartered in Hawthorne, California, the privately held company serves NASA and commercial satellite operators, building most systems in-house.