Senior Data Engineer
NorthBay Solutions5 months ago
Islamabad, PakistanSenior
Responsibilities
- Design, develop, and maintain scalable data pipelines using PySpark, AWS Glue, and Amazon EMR.
- Build and optimize data ingestion, transformation, processing, and ETL/ELT frameworks.
- Develop and maintain enterprise data warehouse solutions using Amazon Redshift.
- Write complex SQL queries, stored procedures, and data transformations for analytics and reporting.
- Tune and optimize Spark jobs, ETL pipelines, SQL queries, and Redshift workloads.
- Ensure data quality, integrity, security, and governance across data platforms.
- Troubleshoot production issues and perform root cause analysis for data incidents.
- Collaborate with stakeholders, analysts, architects, and engineering teams on data requirements.
- Participate in code reviews, technical design discussions, and best-practice implementation.
- Monitor pipelines, identify performance improvements, automate processes, and maintain technical documentation, data models, and operational procedures.
Requirements
- 5–8 years of experience in data engineering and data warehousing.
- Strong hands-on experience with PySpark, AWS Glue, Amazon EMR, Amazon Redshift, SQL, data warehousing, dimensional modeling, and ETL/ELT processes.
- Experience tuning Spark jobs, SQL queries, ETL pipelines, and data warehouse workloads involving large-scale datasets and distributed processing.
- Strong debugging, troubleshooting, analytical, communication, stakeholder management, and collaboration skills.
- Bachelor's degree in Computer Science, Software Engineering, Information Technology, or a related field.
- Relevant AWS certifications are advantageous.
- Experience with additional AWS services, workflow orchestration tools, data governance, security, and compliance practices is preferred.
- Knowledge of CI/CD pipelines, DevOps practices for data platforms, and Agile/Scrum methodologies is preferred.
Benefits
- Hybrid work arrangement in Pakistan.
- Must be available during UAE business hours and follow UAE public holidays.
- Works with a UAE-based customer and distributed, geographically diverse teams.
- Opportunity to work independently in a remote environment.
Tech Stack
Amazon RedshiftSQL
Categories
Data Engineering
About NorthBay Solutions
NorthBay Solutions is a privately held IT services firm that helps enterprises modernize on AWS through cloud migration, data engineering, and AI/ML solutions. An AWS Premier Consulting Partner founded in 2010 and headquartered in Andover, MA, it also partners with Snowflake and provides managed services, DevOps enablement, and application development. The company operates distributed teams across the US, Canada, UAE, India, and Pakistan.