Staff Software Engineer - Ingestion
Databricks1 year ago
Bengaluru, IndiaStaff+
Responsibilities
- Build a highly scalable, available, and fault-tolerant ingestion engine processing hundreds of terabytes daily across thousands of customers.
- Design systems that extract data from OLTP systems while minimizing production-system load using techniques such as incremental data capture and log parsing.
- Perform low-level systems debugging, performance measurement, and optimization on large production clusters.
- Design architecture, influence the product roadmap, and own new technical projects.
- Prevent and investigate production issues using deep systems expertise.
- Plan and lead complex technical projects involving multiple company teams.
- Influence organizational roadmap and direction and lead technical review committees or engineering initiatives.
- Break down complex problems, prototype and validate solutions, and reduce technical risk.
- Mentor engineers and contribute as a Technical Team Lead through sprint planning, delegation, and project planning.
Requirements
- 15+ years of industry experience building and supporting large-scale distributed systems.
- Experience with database replication, backup, and transaction recovery at a major database vendor such as Microsoft SQL Server or Oracle.
- Strong foundation in algorithms and data structures and their real-world applications.
- Ability to work toward a multi-year vision with incremental deliverables.
- Experience driving company initiatives focused on customer satisfaction.
- Motivation to deliver customer value and impact.
Benefits
- Comprehensive employee benefits and perks, with region-specific details available through Databricks' benefits portal.
Tech Stack
Categories
BackendData Engineering
About Databricks
Databricks builds a cloud-based data and AI platform centered on the lakehouse architecture, combining data engineering, analytics, and machine learning with Apache Spark, Delta Lake, and MLflow. It sells subscriptions and cloud services to enterprises that need to unify data pipelines and develop large-scale AI and analytics. Founded in 2013 by the creators of Apache Spark and headquartered in San Francisco, the company is privately held and serves organizations across many industries.