Principal Software Engineer (Data Platform)
NorthBay Solutions2 months ago
Lahore, PakistanStaff+
Responsibilities
- Design and develop scalable ETL/ELT pipelines using Azure Data Factory and Azure Databricks.
- Develop data transformation solutions using PySpark, Spark SQL, and Delta Lake.
- Build and maintain data lakehouse solutions using Azure Data Lake Storage Gen2.
- Use Azure Synapse Analytics, Azure Event Hub, Azure Logic Apps, Azure Function Apps, and Azure SQL Database for data integration and processing.
- Implement secure integrations using Azure Key Vault and Microsoft SharePoint.
- Troubleshoot and optimize data pipelines and Spark workloads.
Requirements
- At least 5 years of hands-on experience designing and developing scalable data solutions on Microsoft Azure.
- Experience with Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Event Hub, Azure Data Lake Storage Gen2, Azure Logic Apps, Azure Function Apps, Azure SQL Database, Azure Key Vault, and Microsoft SharePoint integration.
- Experience with PySpark, Spark SQL, Delta Lake, Delta Tables, and data lakehouse architecture.
Benefits
- Hybrid work arrangement in Pakistan.
- Working hours aligned with UAE business hours.
- Following the UAE public holiday calendar.
Tech Stack
Categories
Data Engineering
About NorthBay Solutions
NorthBay Solutions is a privately held IT services firm that helps enterprises modernize on AWS through cloud migration, data engineering, and AI/ML solutions. An AWS Premier Consulting Partner founded in 2010 and headquartered in Andover, MA, it also partners with Snowflake and provides managed services, DevOps enablement, and application development. The company operates distributed teams across the US, Canada, UAE, India, and Pakistan.