Staff Software Engineer - AI Compute, Together Cloud
Together AI6 days ago
Amsterdam, NetherlandsStaff+
Responsibilities
- Own the GPU and network virtualization stack, including hypervisor, kernel, and software-defined networking work for high-performance and isolated infrastructure.
- Architect and roadmap the in-data-center IaaS layer for compute, storage, networking, virtual machines, filesystems, VPCs, and InfiniBand partitions.
- Lead the build-out of a new Vera Rubin data center from hardware bring-up through customer-facing APIs.
- Design GPU scheduling and global management planes for on-demand and reserved clusters across multiple data centers.
- Architect monitoring, automated detection, isolation, and remediation systems for fault-tolerant distributed compute.
- Set technical direction through design reviews, architectural standards, cross-team alignment, and dependency resolution.
- Mentor engineers, grow team expertise, improve testing and developer tooling, and contribute to the open-source Together AI platform.
Requirements
- 7+ years of professional software development experience and expert proficiency in at least one backend language, with Golang preferred.
- Experience owning the architecture of large distributed systems from initial design through production at scale.
- Deep experience building and operating globally distributed, high-performance microservice architectures across AWS, Azure, or GCP.
- Expert knowledge of compute, networking, storage, concurrency, memory management, performant I/O, and global-scale systems.
- Demonstrated technical leadership through mentoring, design reviews, and alignment across teams without direct reporting relationships.
- Strong written communication, design documentation, diplomacy, troubleshooting, and stakeholder collaboration skills.
- Experience operating reliable customer-facing production systems and owning infrastructure automation, observability, and CI/CD.
- Preferred qualifications include Kubernetes internals, VMs and hypervisors, data-center networking, Cluster API, high-performance computing, GPU or InfiniBand virtualization, IaaS or PaaS, DPUs or SmartNICs, and GPU programming with NCCL or CUDA.
Tech Stack
About Together AI
Together AI builds an AI-native cloud platform for developers, offering high-performance inference, fine-tuning/model shaping, and large-scale pre-training on on-demand GPU clusters with APIs and managed services. It emphasizes open-source models that teams can run and adapt, and also provides infrastructure for decentralized and scalable workloads. Founded in 2022 and headquartered in San Francisco, it is privately held and reports notable customers including Cursor, ElevenLabs, Salesforce, and Zoom.