Principal Software Engineer (Data Platform)
NorthBay Solutions10 days ago
Lahore, PakistanStaff+
Responsibilities
- Design and develop scalable ETL/ELT pipelines using Azure Data Factory and Azure Databricks.
- Develop data transformation solutions using PySpark, Spark SQL, and Delta Lake.
- Build and maintain data lakehouse solutions using Azure Data Lake Storage Gen2.
- Use Azure Synapse Analytics, Azure Event Hub, Azure Logic Apps, Azure Function Apps, and Azure SQL Database for data integration and processing.
- Implement secure integrations using Azure Key Vault and Microsoft SharePoint.
- Troubleshoot and optimize data pipelines and Spark workloads.
Requirements
- At least 5 years of hands-on experience designing and developing scalable data solutions on Microsoft Azure.
- Experience with Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Event Hub, Azure Data Lake Storage Gen2, Azure Logic Apps, Azure Function Apps, Azure SQL Database, Azure Key Vault, and Microsoft SharePoint integration.
- Experience with PySpark, Spark SQL, Delta Lake, Delta Tables, and data lakehouse architecture.
Benefits
- Hybrid work arrangement in Pakistan.
- Working hours aligned with UAE business hours.
- Following the UAE public holiday calendar.
Tech Stack
Categories
Data Engineering