Staff Software Engineer, GPU Infrastructure Lifecycle Management
Together AI3 months ago
Base Salary
$240k - $280k/yr
Responsibilities
- Design and implement versioned software state machines for the full physical-host lifecycle, from discovery and GPU driver/CUDA bring-up through health validation and decommissioning or RMA.
- Build declarative self-service APIs and a control plane that let inference teams provision, scale, and tear down clusters without manual intervention.
- Automate detection, draining, repair or replacement, and reintegration of degraded or failed nodes.
- Ensure pipeline reliability through idempotency, retries, rollback, and drift detection.
- Partner with inference and ML platform teams to encode cluster shapes, topology, interconnects, and scheduling constraints as platform abstractions.
- Operate and support the production platform while maintaining strong typing, automated tests, versioning, code review, and deployment practices.
Requirements
- Strong software engineering background in Go, Python, Rust, or similar, including writing and testing production software.
- Experience with durable workflow orchestration tools such as Temporal, Cadence, or equivalent.
- Experience building software control planes or orchestration systems that model state and continuously reconcile it, such as Kubernetes controllers or operators.
- Experience designing event-driven systems using message queues, event streams, or pub/sub rather than polling or cron-driven scripts.
- Experience building internal platforms or APIs consumed by other engineering teams with attention to developer experience.
- Exposure to bare-metal provisioning technologies, networking fundamentals, or GPU and accelerator infrastructure is preferred.
- Experience with GPU cluster software stacks such as NCCL, CUDA, and InfiniBand/RoCE is preferred.
- Prior experience at a hyperscaler, GPU cloud, or datacenter-scale infrastructure organization is preferred.
- Systems programming experience in Rust or Go is preferred.
Benefits
- Startup equity
- Health insurance
- Other competitive benefits
- Full-time position in the US
Tech Stack
About Together AI
Together AI builds an AI-native cloud platform for developers, offering high-performance inference, fine-tuning/model shaping, and large-scale pre-training on on-demand GPU clusters with APIs and managed services. It emphasizes open-source models that teams can run and adapt, and also provides infrastructure for decentralized and scalable workloads. Founded in 2022 and headquartered in San Francisco, it is privately held and reports notable customers including Cursor, ElevenLabs, Salesforce, and Zoom.