Senior Software Engineer, Compute Infrastructure
Databricks9 hours ago
Base Salary
$164k - $205k/yr
Responsibilities
- Design and build the next generation of the Serverless platform with high availability, low latency, and scale-to-zero efficiency.
- Build a unified compute stack for heterogeneous workloads, including GPU instance qualification for AI training and stateful execution for Lakebase.
- Develop platform-level abstractions that operate across AWS, Azure, and GCP.
- Optimize systems from the API gateway through host-level networking and container orchestration.
- Build infrastructure and tools from the ground up to meet large-scale production requirements.
Requirements
- At least 5 years of production-level experience with Python, Java, Scala, C++, or a similar language.
- At least 4 years of experience developing large-scale distributed systems from scratch.
- Experience with Kubernetes, Docker, or MicroVM technologies such as Firecracker.
- Strong understanding of AWS, Azure, and GCP internals.
- Ability to build foundational tools and systems when existing solutions do not operate at the required scale.
Benefits
- Comprehensive benefits and perks are offered, with specific details varying by region.
Tech Stack
About Databricks
Databricks builds a cloud-based data and AI platform centered on the lakehouse architecture, combining data engineering, analytics, and machine learning with Apache Spark, Delta Lake, and MLflow. It sells subscriptions and cloud services to enterprises that need to unify data pipelines and develop large-scale AI and analytics. Founded in 2013 by the creators of Apache Spark and headquartered in San Francisco, the company is privately held and serves organizations across many industries.