Principal Software Engineer (Data Platform)
NorthBay Solutions10 days ago
Islamabad, PakistanStaff+
Responsibilities
- Design and develop scalable ETL/ELT pipelines using Azure Data Factory and Azure Databricks.
- Develop data transformation solutions using PySpark, Spark SQL, and Delta Lake.
- Build and maintain data lakehouse solutions using Azure Data Lake Storage Gen2.
- Work with Azure Synapse Analytics, Azure Event Hub, Azure Logic Apps, Azure Function Apps, and Azure SQL Database for data integration and processing.
- Implement secure integrations using Azure Key Vault and Microsoft SharePoint.
- Troubleshoot and optimize data pipelines and Spark workloads.
Requirements
- At least 5 years of hands-on experience designing and developing scalable data solutions on Microsoft Azure.
- Experience with Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Event Hub, Azure Data Lake Storage Gen2, Azure Logic Apps, Azure Function Apps, Azure SQL Database, Azure Key Vault, and Microsoft SharePoint integration.
- Proficiency with PySpark, Spark SQL, Delta Lake, and Delta Tables.
- Experience with data lakehouse architecture.
- Ability to work UAE business hours and follow the UAE public holiday calendar.
Benefits
- Hybrid work arrangement in Pakistan.
- Working hours aligned with UAE business hours.
- Following the UAE public holiday calendar.
Tech Stack
AzureDatabricks
Categories
Data Engineering