Principal Software Engineer (Data Platform)
NorthBay Solutions10 days ago
Karachi, PakistanStaff+
Responsibilities
- Design and develop scalable ETL/ELT pipelines using Azure Data Factory and Azure Databricks.
- Develop data transformation solutions using PySpark, Spark SQL, and Delta Lake.
- Build and maintain data lakehouse solutions using Azure Data Lake Storage Gen2.
- Integrate Synapse Analytics, Event Hub, Logic Apps, Function Apps, and Azure SQL Database for data processing.
- Implement secure integrations using Azure Key Vault and Microsoft SharePoint.
- Troubleshoot and optimize data pipelines and Spark workloads.
Requirements
- 5+ years of hands-on experience designing and developing scalable data solutions on Microsoft Azure.
- Experience with Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Event Hub, Azure Data Lake Storage Gen2, Azure Logic Apps, Azure Function Apps, Azure SQL Database, Azure Key Vault, Microsoft SharePoint integration, PySpark, Spark SQL, Delta Lake, and data lakehouse architecture.
- Comfort working UAE business hours and following the UAE public holiday calendar.
Benefits
- Hybrid work arrangement in Pakistan.
- Working hours aligned with UAE business hours.
- Following the UAE public holiday calendar.
Tech Stack
AzureDatabricks
Categories
Data Engineering