AI Infrastructure System Engineer Bangalore
Together AI19 hours ago
Bengaluru, IndiaMid Level
Responsibilities
- Design and build fleet automation systems for provisioning, validating, deploying, upgrading, repairing, and retiring GPU clusters.
- Develop AI infrastructure agents for deployment automation, root-cause analysis, incident triage, and autonomous remediation.
- Build fleet intelligence platforms that monitor hardware, firmware, networking, storage, thermals, and workload performance to predict failures.
- Develop software that improves GPU availability, utilization, performance, and reliability across thousands of accelerators.
- Create automated validation systems for GPUs, InfiniBand/RoCE fabrics, NVLink/NVSwitch, storage, and distributed AI workloads.
- Build internal platforms and developer tools for software-driven infrastructure management.
- Improve deployment velocity, reliability, and operational efficiency through automation.
- Collaborate with hardware, networking, platform, and AI teams.
Requirements
- At least 3 years of experience building distributed systems, infrastructure platforms, or large-scale backend software.
- Strong software engineering skills in Python, Go, or Rust.
- Experience building platforms, automation systems, or developer infrastructure.
- Experience with Linux, Kubernetes, Terraform, Ansible, or similar infrastructure technologies.
- Ability to understand complex problems across hardware and software.
- Passion for solving infrastructure challenges through software and an automation-first mindset.
- Preferred experience includes GPU infrastructure, CUDA, NCCL, NVLink/NVSwitch, InfiniBand or RoCE networking, bare-metal provisioning, large-scale AI clusters, hardware health monitoring, distributed storage, and AI agents.
About Together AI
Together AI builds an AI-native cloud platform for developers, offering high-performance inference, fine-tuning/model shaping, and large-scale pre-training on on-demand GPU clusters with APIs and managed services. It emphasizes open-source models that teams can run and adapt, and also provides infrastructure for decentralized and scalable workloads. Founded in 2022 and headquartered in San Francisco, it is privately held and reports notable customers including Cursor, ElevenLabs, Salesforce, and Zoom.