Staff Software Engineer - Streaming
Databricks2 months ago
Base Salary
$182k - $247k/yr
Responsibilities
- Set and drive the technical vision for Spark Structured Streaming across open source and the Databricks Data Intelligence Platform.
- Design and build core engine capabilities including state management, new operators, and architectural improvements.
- Improve latency, throughput, cost, engineering quality, and production operability.
- Drive stream-processing adoption across the Databricks product portfolio.
- Guide long-term architecture and technical-debt decisions in balance with the product roadmap.
- Mentor and technically lead engineers and partner with the Engineering Manager on hiring and talent development.
Requirements
- BS or higher in Computer Science or a related technical field, or equivalent practical experience.
- 8+ years building related systems in big-data ecosystems, Apache Spark, database internals, or similar areas.
- Passion for database systems, storage systems, distributed systems, language design, or performance optimization.
- Track record of delivering features while maintaining strong operational excellence and engineering quality.
- Ability to work toward a multi-year vision with incremental deliverables.
- Ability to collaborate cross-functionally with product management and directly with customers while understanding product and customer personas.
Benefits
- Comprehensive employee benefits and perks, with region-specific details available from Databricks.
- Eligibility for annual performance bonus and equity may be included in the total compensation package.
Tech Stack
Categories
BackendData Engineering
About Databricks
Databricks builds a cloud-based data and AI platform centered on the lakehouse architecture, combining data engineering, analytics, and machine learning with Apache Spark, Delta Lake, and MLflow. It sells subscriptions and cloud services to enterprises that need to unify data pipelines and develop large-scale AI and analytics. Founded in 2013 by the creators of Apache Spark and headquartered in San Francisco, the company is privately held and serves organizations across many industries.