Principal Software Engineer (Data Platform)
NorthBay Solutions2 months ago
Karachi, PakistanStaff+
Responsibilities
- Design and develop scalable ETL/ELT pipelines using Azure Data Factory and Azure Databricks.
- Develop data transformation solutions using PySpark, Spark SQL, and Delta Lake.
- Build and maintain data lakehouse solutions using Azure Data Lake Storage Gen2.
- Integrate Synapse Analytics, Event Hub, Logic Apps, Function Apps, and Azure SQL Database for data processing.
- Implement secure integrations using Azure Key Vault and Microsoft SharePoint.
- Troubleshoot and optimize data pipelines and Spark workloads.
Requirements
- 5+ years of hands-on experience designing and developing scalable data solutions on Microsoft Azure.
- Experience with Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Event Hub, Azure Data Lake Storage Gen2, Azure Logic Apps, Azure Function Apps, Azure SQL Database, Azure Key Vault, Microsoft SharePoint integration, PySpark, Spark SQL, Delta Lake, and data lakehouse architecture.
- Comfort working UAE business hours and following the UAE public holiday calendar.
Benefits
- Hybrid work arrangement in Pakistan.
- Working hours aligned with UAE business hours.
- Following the UAE public holiday calendar.
Tech Stack
AzureDatabricks
Categories
Data Engineering
About NorthBay Solutions
NorthBay Solutions is a privately held IT services firm that helps enterprises modernize on AWS through cloud migration, data engineering, and AI/ML solutions. An AWS Premier Consulting Partner founded in 2010 and headquartered in Andover, MA, it also partners with Snowflake and provides managed services, DevOps enablement, and application development. The company operates distributed teams across the US, Canada, UAE, India, and Pakistan.