Data Engineer
ADA Global25 days ago
Bengaluru, IndiaMid Level / Senior
Responsibilities
- Design, implement, optimize, and maintain scalable ETL/ELT data pipelines for structured and unstructured data.
- Integrate internal and external data sources into centralized systems and build batch and real-time processing workflows.
- Develop and maintain data lake, data warehouse, or data mesh architectures with strong availability, performance, and scalability.
- Apply data modeling best practices for analytics and machine learning use cases.
- Write maintainable software in Python, Java, or Scala and build reusable data services and APIs.
- Implement unit testing, integration testing, and CI/CD pipelines for data workflows.
- Implement data validation, monitoring, and observability to ensure data accuracy and reliability.
- Enforce data security, compliance, and privacy standards and collaborate on data definitions, lineage, and documentation.
Requirements
- Bachelor’s or master’s degree in Computer Science, Engineering, or a related field.
- 2 –4 years of experience in data engineering, software engineering, or backend systems.
- Strong programming skills in Python, Java, or Scala.
- Experience with distributed data processing frameworks such as Apache Spark, Flink, or Beam.
- Proficiency with SQL and relational databases, plus familiarity with Cassandra, MongoDB, or DynamoDB.
- Hands-on experience with AWS, GCP, or Azure and data services such as Redshift, BigQuery, Snowflake, or Databricks.
- Familiarity with Airflow, Luigi, or Dagster and CI/CD pipelines.
- Understanding of software engineering best practices, including version control, testing, and code reviews.
Tech Stack
Amazon DynamoDBAmazon RedshiftApache AirflowApache BeamApache CassandraApache FlinkApache SparkAWSAzureDatabricksGoogle BigQueryGoogle Cloud PlatformJavaMongoDBPythonScalaSnowflakeSQL
Categories
Data Engineering