Software Development Engineer II - Data

3 months ago

Bengaluru, IndiaMid Level

Responsibilities

Architect and implement high-scale streaming and batch data pipelines with Apache Spark, Flink, and Airflow.
Build and maintain data lakes, ingestion frameworks, and lakehouse solutions using Snowflake, Apache Iceberg, and Parquet.
Design data models for structured and semi-structured analytical and operational datasets.
Develop low-latency pipelines using Kafka and Spark Structured Streaming that support billions of events per day.
Automate and orchestrate end-to-end ELT workflows with reliability, observability, and failure recovery.
Build scalable AWS-based data infrastructure using services such as S3, Lambda, and ECS.
Implement data quality checks, monitoring, observability, and performance tuning.
Collaborate with Product, Design, and cross-functional stakeholders on data platforms powering analytics, AI, and real-time decision-making.

Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.
At least 3 years of data engineering experience designing large-scale distributed data systems.
Strong expertise with Snowflake and other distributed analytical data stores.
Hands-on experience with Apache Spark, Flink, Airflow, Iceberg, and Parquet.
Strong understanding of data modeling, schema design, query optimization, and partitioning strategies at scale.
Proficiency in Python, SQL, Scala, and Go or Node.js, with strong debugging and performance-tuning skills.
Experience with streaming architectures, CDC pipelines, and data observability frameworks.
Experience deploying containerized applications with Docker, Kubernetes, or ECS.
Familiarity with AI coding assistants such as Cursor, Claude Code, or GitHub Copilot.
Preferred exposure to CI/CD pipelines, automated testing, and infrastructure-as-code for data workflows.
Preferred familiarity with Kafka, Kinesis, Pulsar, Druid, Pinot, or Rockset.
Understanding of data governance, lineage tracking, and compliance requirements in a multi-tenant SaaS platform.

1,001-5,000 employees