2 months ago
Remote, IndiaMid Level / Senior
Responsibilities
- Design and develop core components of the data platform, including high-volume event ingestion pipelines.
- Develop reusable data access layers and ETL/ELT processes for transforming data into Lakehouse assets.
- Contribute to the development of internal APIs and backend services for data platform interaction.
- Build robust, fault-tolerant data pipelines focusing on correctness and availability.
- Implement schema enforcement and validation gates for data governance.
- Optimize code and query performance for efficiency.
- Build and own the observability framework for monitoring pipeline health.
- Identify and resolve production bottlenecks and participate in incident response.
- Apply automated testing to ensure long-term maintainability.
- Participate in design discussions and code reviews.
Requirements
- 3+ years of experience in software engineering with a focus on backend or data systems.
- Strong hands-on experience with Apache Spark and the Databricks/Delta Lake ecosystem.
- Expert-level skills in Python and SQL with a deep understanding of software design principles.
- Experience with CI/CD practices and knowledge of Terraform or similar IaC tools.
- Background in building monitored systems with a focus on data quality.
Benefits
- Generous paid parental leave.
- Flexible time off.
- Medical, dental, and vision insurance.
- Sabbatical after 5 years.