Senior Software Engineer - Ingestion
Databricks2 months ago
Bengaluru, IndiaSenior
Responsibilities
- Build and operate a highly scalable, available, and fault-tolerant ingestion engine processing hundreds of terabytes of data daily across thousands of customers.
- Develop systems that extract data from OLTP systems while minimizing load on production databases using incremental data capture and log parsing techniques.
- Perform low-level systems debugging, performance measurement, and optimization on large production clusters.
- Design architectures, influence the product roadmap, and own new technical projects.
- Investigate and help prevent production issues across the full development and operations lifecycle.
- Plan and lead complex technical projects involving multiple internal teams.
- Prototype and validate solutions to break down complex technical problems and reduce risk.
Requirements
- Bachelor’s degree or higher in computer science or a related field.
- At least five years of production-level experience with Python, Java, Scala, C++, or a similar language.
- Experience developing large-scale distributed systems from scratch.
- Hands-on experience developing and operating backend systems.
- Ability to contribute across design, development, implementation, and ongoing operations.
- Experience with database replication, backup, or transaction recovery at a major database vendor such as Microsoft SQL Server, Oracle, or IBM is preferred.
Benefits
- Databricks offers comprehensive employee benefits and perks, with details varying by region.
- The company emphasizes an inclusive culture and equal employment opportunity.
About Databricks
Databricks builds a cloud-based data and AI platform centered on the lakehouse architecture, combining data engineering, analytics, and machine learning with Apache Spark, Delta Lake, and MLflow. It sells subscriptions and cloud services to enterprises that need to unify data pipelines and develop large-scale AI and analytics. Founded in 2013 by the creators of Apache Spark and headquartered in San Francisco, the company is privately held and serves organizations across many industries.