about 4 hours ago
Bengaluru, IndiaStaff+ / Senior
H1B Sponsor
Responsibilities
- Design and evolve a multi-tenant, cloud-scale data platform.
- Drive the transition from batch-oriented to real-time data processing.
- Implement data ingestion pipelines for batch, streaming, and CDC.
- Ensure strong tenant isolation, governance, and data quality standards.
- Collaborate with cross-functional teams to support analytical products.
Requirements
- 7+ years of experience in Data Engineering.
- Strong expertise in Python, SQL, and Apache Spark.
- Experience building scalable batch and real-time ETL/ELT pipelines.
- Hands-on experience with AWS services including EMR, S3, Glue, and Athena.
- Experience with Kafka, Flink, or Kinesis for streaming data processing.
- Strong knowledge of dimensional modeling, Data Vault, and data warehousing concepts.
- Experience with Delta Lake, Apache Iceberg, or Apache Hudi.
- Expertise in workflow orchestration using Airflow.
- Experience implementing data quality frameworks and monitoring solutions.
- Strong understanding of partitioning, schema evolution, and performance optimization.
- Familiarity with CI/CD, Git, and Infrastructure as Code tools is a plus.
Tech Stack
Categories
AI & MLData Engineering