Principal Engineer, Compute Fleet Management
Databricks9 months ago
Base Salary
$264k - $322k/yr
Responsibilities
- Provision and pool billions of cloud resources across AWS, Azure, and GCP for workload performance and efficiency.
- Architect horizontally scalable systems resilient to zonal and cloud account-level failures.
- Lead development of the lowest-dependency systems used to bootstrap and manage Databricks' compute platform.
- Drive 99.99% availability for batch and serving workloads.
- Increase compute utilization to 60% or higher while maintaining tolerance for cloud failures.
- Architect and enforce strong security and performance isolation across diverse customer workloads.
- Own complex strategic engineering initiatives and coordinate execution across teams and organizational boundaries.
Requirements
- Seasoned Principal Engineer with a track record of building and operating large-scale, mission-critical infrastructure systems in production.
- Deep hands-on experience developing and operating high-scale distributed systems on at least one major public cloud.
- Experience leading complex, cross-team, cross-layer, multi-quarter strategic engineering initiatives from concept through execution.
- Proven ability to establish technical direction, drive consensus, and lead large technical efforts across organizational boundaries.
- Exceptional planning, project-tracking, and management of complex cross-organizational dependencies.
- Experience managing and scaling a massive GPU fleet for AI/ML workloads is highly desirable.
- Experience developing and operating large-scale distributed systems across AWS, Azure, and GCP is highly desirable.
Benefits
- Comprehensive benefits and perks, with region-specific details available through Databricks' benefits site.
Tech Stack
About Databricks
Databricks is the Data and AI company. More than 20,000 organizations worldwide — including adidas, AT&T, Bayer, Block, Mastercard, Rivian, Unilever, and over 60% of the Fortune 500 — rely on Databricks to build and scale data and AI apps, analytics and agents. Headquartered in San Francisco with 30+ offices around the globe, Databricks offers a unified Data Intelligence Platform that includes Agent Bricks, Lakeflow, Lakehouse, Lakebase and Unity Catalog. --- Databricks applicants Please apply through our official Careers page at databricks.com/company/careers. All official communication from Databricks will come from email addresses ending with @databricks.com or @goodtime.io (our meeting tool).