Staff Software Engineer - Ingestion
Databricks2 months ago
Bengaluru, IndiaStaff+
Responsibilities
- Build and operate a highly scalable, available, and fault-tolerant data-ingestion engine processing hundreds of terabytes daily across thousands of customers.
- Develop systems for extracting data from OLTP systems using techniques such as incremental data capture and log parsing while minimizing production load.
- Perform low-level systems debugging, performance measurement, and optimization on large production clusters.
- Design architecture, own new projects, influence the product roadmap, and collaborate with multiple product and engineering teams.
- Prevent and investigate production issues using deep systems experience.
- Plan and lead complicated technical projects spanning several internal teams.
- Drive engineering initiatives and participate in technical review committees that influence organizational direction.
- Break down complex problems, prototype and validate solutions, and reduce technical risk.
- Mentor engineers, lead sprint planning, delegate assignments, and contribute to project planning as a Technical Team Lead.
Requirements
- 15+ years of industry experience building and supporting large-scale distributed systems.
- Experience with database replication, backup, and transaction recovery at a major database vendor such as Microsoft SQL Server, Oracle, or IBM.
- Strong foundation in algorithms and data structures and their practical applications.
- Ability to work toward a multi-year vision while delivering incremental improvements.
- Experience driving company initiatives focused on customer satisfaction.
- Motivation to deliver customer value and measurable impact.
Benefits
- Databricks offers comprehensive benefits and perks, with region-specific details provided separately.
- The company is committed to an inclusive workplace and equal employment opportunity.
About Databricks
Databricks builds a cloud-based data and AI platform centered on the lakehouse architecture, combining data engineering, analytics, and machine learning with Apache Spark, Delta Lake, and MLflow. It sells subscriptions and cloud services to enterprises that need to unify data pipelines and develop large-scale AI and analytics. Founded in 2013 by the creators of Apache Spark and headquartered in San Francisco, the company is privately held and serves organizations across many industries.