Senior Staff Software Engineer - Lakeflow Pipelines Datasets
Databricks6 months ago
Base Salary
$229k - $314k/yr
Responsibilities
- Architect and deliver a scalable, fault-tolerant platform for declarative pipelines and materialized views.
- Translate ambiguous large-scale technical challenges into incremental execution plans suitable for exabyte-scale systems.
- Bring research and internal R&D ideas into production systems that solve customer problems.
- Lead performance diagnosis, low-level debugging, and optimization on large production clusters.
- Shape product direction and own technical strategy from design through delivery.
- Drive complex, multi-team technical initiatives with sustained coordination and technical judgment.
Requirements
- 10+ years of experience building, operating, and evolving large-scale distributed systems in production.
- Deep expertise in database internals, storage systems, distributed computing, streaming systems, language/API design, or performance engineering.
- Experience executing a multi-year technical vision through incremental milestones.
- Strong algorithmic foundations and practical engineering judgment.
- Experience building cross-team alignment and driving initiatives from conception through customer adoption.
- BS, MS, or PhD in Computer Science or a related field, or equivalent depth of experience.
Benefits
- Comprehensive employee benefits and perks, with details varying by region.
Tech Stack
Apache SparkMLflow
Categories
BackendData Engineering
About Databricks
Databricks builds a cloud-based data and AI platform centered on the lakehouse architecture, combining data engineering, analytics, and machine learning with Apache Spark, Delta Lake, and MLflow. It sells subscriptions and cloud services to enterprises that need to unify data pipelines and develop large-scale AI and analytics. Founded in 2013 by the creators of Apache Spark and headquartered in San Francisco, the company is privately held and serves organizations across many industries.