7 days ago
Bengaluru, IndiaStaff+
Responsibilities
- Define and evolve architectures for scalable data platforms, pipelines, and analytical data systems.
- Design and build resilient batch and streaming pipelines using Python, SQL, and distributed processing frameworks.
- Lead cloud data platform design and operation across GCP and AWS, including BigQuery, Cloud Storage, Dataflow, S3, Redshift, and Glue.
- Establish maintainable, version-controlled, and testable data transformation practices and analytics-ready data models.
- Design ingestion patterns for SaaS applications, databases, APIs, and other sources, including schema-change and source-reliability standards.
- Design, develop, and optimize Apache Spark workloads using PySpark or Scala.
- Guide workflow orchestration, monitoring, error handling, CI/CD, automated testing, and release processes for data systems.
- Define data validation, lineage, schema evolution, access control, and secure data-handling practices.
- Lead design reviews, mentor engineers, and resolve complex cross-system problems.
- Partner with backend, analytics, product, and business teams to translate data needs into platform capabilities.
- Establish service health measures and support practices, including incident response, root-cause analysis, and recurrence prevention.
Requirements
- 8+ years of experience in software or data engineering, including substantial hands-on experience building production data systems.
- Experience designing and operating scalable data platforms or distributed data processing systems.
- Strong programming skills in Python and SQL.
- Practical Apache Spark experience using PySpark or Scala.
- Experience with at least one major cloud platform, GCP or AWS, and its data services.
- Experience with workflow orchestration tools such as Apache Airflow and modern CI/CD practices.
- Experience developing data models and transformation workflows; familiarity with dbt is preferred.
- Experience building ingestion frameworks for APIs, SaaS platforms, databases, or event streams; familiarity with dlt is preferred.
- Strong understanding of data quality, schema evolution, lineage, security, and production reliability.
- Ability to lead technical design across teams, mentor engineers, and communicate trade-offs clearly.
- Track record of balancing hands-on delivery with longer-term technical direction.
Benefits
- Competitive compensation, equity, bonuses, and comprehensive benefits.
- Flexible time off, holidays, sick time, health and wellness initiatives, and a charitable match program.
- Employee discounts, transportation reimbursements, and fitness facilities, varying by location.
- Flexible hybrid work environment combining in-office collaboration with work-from-home benefits.
Tech Stack
Categories
Data Engineering