3 days ago
Remote, CanadaSenior
Responsibilities
- Develop utilities for migration, storage, and processing using Azure Synapse Analytics.
- Design and implement cloud migration strategies for cloud and on-premise applications.
- Troubleshoot and optimize complex distributed systems within the Azure ecosystem, especially Azure Synapse environments.
- Build ingestion tools and processing jobs for terabytes- or petabytes-per-day workloads while optimizing performance and cost.
- Design and develop large-scale storage and compute solutions for customers using Azure Synapse capabilities.
- Collaborate with Infrastructure, Network, Engineering, DevOps, SiteOps, cloud customers, and cross-specialty architects.
- Build automation, testing, monitoring, administration, and data operations tooling across Azure Synapse workspaces and associated cloud clusters.
- Lead or support technology teams, mentor junior team members, and communicate technical plans to technical and non-technical stakeholders.
Requirements
- At least 3 years of hands-on data engineering experience with data structures, distributed systems, data modeling, and databases.
- Strong proficiency in SQL and hands-on experience with Python and PySpark.
- Extensive experience developing cloud-based big data migration, storage, and processing solutions.
- Experience building and supporting large-scale production systems.
- Strong experience developing ETL/ELT pipelines with Azure Data Factory and Synapse Pipelines.
- Deep expertise in Azure, particularly Azure Synapse Analytics, SQL Pools, Spark Pools, Data Explorer, and Pipelines.
- Experience with Azure Data Lake Storage Gen2 and Apache Spark, especially within Azure Synapse Spark Pools.
- Experience with Azure Synapse SQL Pools, Azure Cosmos DB, Azure Data Explorer, and other Azure data services.
- Experience with Azure Data Factory, Apache Airflow, or similar orchestration frameworks.
- Experience with Azure Event Hubs, Azure Service Bus, and Kafka.
- Ability to lead technology teams, mentor others, and lead multiple large projects.
- Preferred experience with insurance data or domain knowledge.
- Preferred experience with Apache Airflow, Databricks, dbt, Microsoft Fabric, Snowflake, Kubernetes, or Azure Kubernetes Service.
- Preferred knowledge of Terraform, infrastructure as code, Azure DevOps, data governance, data quality, data lineage, and metadata management.
- Strong communication skills and ability to explain complex subjects and technical plans clearly.
Benefits
- Remote work location in Canada.
- Opportunities to work on complex AI, ML, data, and cloud technologies with Fortune 500 companies.
- Upskilling and professional growth opportunities in a research-focused organization.
- Opportunity to work alongside talented colleagues at a rapidly growing AI-first digital engineering company.
- Exposure to innovative projects and a company with more than 60 patents filed.
Tech Stack
Categories
Data Engineering
About Quantiphi
Quantiphi is an AI-first digital engineering and consulting firm that designs and implements machine learning, data, and cloud solutions for large enterprises across industries. Its teams build production systems—such as generative AI applications, computer vision, and predictive analytics—primarily on Google Cloud and other hyperscalers, delivered as professional services and managed solutions. Founded in 2013 and headquartered in Marlborough, Massachusetts, the company is privately held and a Google Cloud partner.