Data Engineer II

3 months ago

Pune, IndiaMid Level

Responsibilities

Design, develop, and maintain scalable ETL/ELT pipelines for structured and unstructured data.
Own assigned data pipeline domains from requirements and design through deployment, monitoring, and improvement.
Implement and evolve data models supporting analytics, BI, data science, and machine learning workloads.
Build and maintain orchestration workflows using Apache Airflow, dbt, or Azure Data Factory.
Optimize pipeline and query performance across Snowflake and Azure Synapse.
Define data quality rules, automated testing, and alerting for reliable data outputs.
Contribute to architectural discussions, platform decisions, and engineering standards.
Collaborate with Data Scientists, Analytics Engineers, and business stakeholders on data requirements.
Conduct code reviews and provide practical guidance to Associate and Data Engineer I engineers.
Maintain documentation for pipelines, data models, and infrastructure components.

Bachelor's degree in Computer Science, Computer Engineering, Information Technology, or a related field.
Four to six years of hands-on experience in data engineering or a closely related software engineering role.
Strong proficiency in SQL and Python for data transformation, automation, and pipeline development.
Working knowledge of data orchestration and transformation tools such as Apache Airflow and dbt.
Understanding of dimensional modeling, star and snowflake schemas, and medallion/lakehouse architecture patterns.
Familiarity with batch and streaming data processing concepts such as Spark, Kafka, and Azure Event Hubs.
Good understanding of software engineering fundamentals, Git, CI/CD practices, and automated testing.
Demonstrable experience delivering production-grade data pipelines in a cloud environment.
Hands-on experience with cloud data platforms on Azure, AWS, or GCP, including services such as Azure Data Factory, Synapse Analytics, S3, Redshift, and BigQuery.
Experience with data warehousing platforms such as Snowflake or Azure Synapse, including query optimization and access control.
Ability to communicate technical designs clearly and provide constructive feedback to junior team members.
Fluent in English; ITIL Certification is advantageous but not mandatory.