1 day ago
Base Salary
$174k - $252k/yr
Responsibilities
- Design and operate globally distributed systems handling millions of queries across multiple regions.
- Build distributed scheduling, dynamic reservation, and fair-sharing systems tuned for token-by-token streaming, bursty agent loops, speculative decoding, and prompt caching.
- Build self-healing systems with failover, fast coordination, and telemetry to achieve 99.999% availability.
- Set architectural direction for token orchestration and distribution across Google Cloud.
- Develop quota and rate-limiting infrastructure for LLM token management and backend overload protection.
Requirements
- Bachelor’s degree or equivalent practical experience.
- At least 5 years of software development experience in one or more programming languages.
- At least 3 years of experience testing, maintaining, or launching software products.
- At least 1 year of experience with software design and architecture.
- At least 3 years of experience developing large-scale infrastructure, distributed systems or networks, or working with compute technologies, storage, or hardware architecture.
- Preferred: master’s degree or PhD in Computer Science or a related technical field.
- Preferred: 5 years of experience with data structures and algorithms.
- Preferred: 1 year of experience in a technical leadership role.
- Preferred: experience with Java or C/C++.
- Preferred: experience in Generative AI.
Benefits
- 15% bonus target, equity, and benefits are offered in addition to base pay.
- The role is part of Google Cloud’s enterprise-grade cloud infrastructure organization.
About Google
A problem isn't truly solved until it's solved for all. Googlers build products that help create opportunities for everyone, whether down the street or across the globe. Bring your insight, imagination and a healthy disregard for the impossible. Bring everything that makes you unique. Together, we can build for everyone. Check out our career opportunities at goo.gle/3DLEokh