Software Development Engineer II - Data Engineering

2 hours ago

Bengaluru, IndiaMid Level

Responsibilities

Implement features for data ingestion, processing, governance, observability, and storage systems.
Build components for scalable and accurate real-time event collection and data pipelines.
Develop pipelines supporting AI model training and autonomous AI agents while maintaining data quality and reliability.
Write clean, maintainable, and well-tested code according to engineering standards.
Improve existing systems and services using modern tooling and engineering practices.
Identify and implement performance optimizations for ingestion, storage, and processing components.

2–4 years of software engineering experience, including at least 2 years focused on data engineering or related fields.
Hands-on experience with data processing frameworks such as Apache Spark, Apache Flink, or similar technologies.
Experience with messaging and stream-processing platforms such as Apache Kafka, Kinesis, or equivalent.
Working knowledge of AWS, GCP, or Azure and their data services, with preference for AWS data services.
Understanding of data warehousing and data lake technologies such as Snowflake, BigQuery, Redshift, or Delta Lake.
Experience building ETL/ELT pipelines with Apache Airflow, Prefect, or similar orchestration systems.
Solid SQL skills and experience with relational and NoSQL databases.
Experience with monitoring and observability tools such as Datadog, Grafana, or Prometheus.
Familiarity with Infrastructure as Code tools such as Terraform or CloudFormation.
Proficiency in Python, Java, Scala, or Go.
Basic understanding of distributed systems concepts and data processing patterns.
Experience with machine-learning pipelines, feature engineering, or model training is preferred.

Amazon DynamoDBAmazon RedshiftApache AirflowApache CassandraApache FlinkApache Kafka Apache Spark AWS AzureDatadogGoGoogle BigQueryGoogle Cloud PlatformGrafanaJavaMySQLPostgreSQLPrometheusPythonScalaSnowflakeSQL Terraform

501-1,000 employees

We’ve moved! 🌟 Follow us on @JioHotstar for all the latest stories, updates & more.