Sr. Software Engineer - Ingestion Core team
Databricks4 hours ago
Base Salary
$166k - $225k/yr
Responsibilities
- Build distributed infrastructure for high-volume ingestion from cloud storage, databases, and file sources.
- Support streaming ingestion, incremental processing, replication, schema evolution, and change data capture into Delta Lake.
- Optimize streaming and distributed workloads for throughput, latency, reliability, scale, and cost.
- Develop customer-facing and internal monitoring and observability capabilities for ingestion workflows.
- Explore and apply ML techniques to optimize streaming workloads.
- Collaborate with partner teams on RAG and AI-agent use cases.
- Build agentic SDKs and define strong evaluations.
Requirements
- 5+ years of experience writing production code in Java, Scala, Go, C++, or Python.
- Experience architecting, developing, and deploying large-scale distributed and asynchronous systems.
- Experience with distributed systems, streaming, Spark, databases, data processing, or change data capture.
- Experience operating systems where scale, throughput, latency, reliability, and cost are important considerations.
- Comfort using AI tools and defining effective evaluations.
Benefits
- Annual performance bonus eligibility may be available.
- Equity eligibility may be included in the compensation package.
- Comprehensive regional benefits and perks are offered.
Tech Stack
Categories
BackendData Engineering
About Databricks
Databricks builds a cloud-based data and AI platform centered on the lakehouse architecture, combining data engineering, analytics, and machine learning with Apache Spark, Delta Lake, and MLflow. It sells subscriptions and cloud services to enterprises that need to unify data pipelines and develop large-scale AI and analytics. Founded in 2013 by the creators of Apache Spark and headquartered in San Francisco, the company is privately held and serves organizations across many industries.