29 days ago
Sydney, Australia +2 moreSenior
Responsibilities
- Design and develop scalable data pipelines and ELT/ETL processes.
- Build and maintain data platforms using Microsoft Fabric and Azure services.
- Develop Lakehouse, Data Warehouse, and Data Factory solutions in Microsoft Fabric.
- Implement data integration across cloud and on-premises data sources.
- Design and optimize data models for analytics, reporting, and AI initiatives.
- Develop and maintain data transformations using SQL, Spark, Python, and PySpark.
- Implement data quality, governance, and security controls.
- Monitor and optimize platform performance, reliability, and cost efficiency.
- Collaborate with architects, analysts, data scientists, and business stakeholders.
- Support CI/CD implementation and automated deployment practices.
Requirements
- At least 5 years of experience in data engineering, data warehousing, or cloud data platform development.
- Strong experience with Microsoft Azure data services, including Azure Data Factory, Azure Data Lake Storage, Azure Synapse Analytics, Azure SQL, Azure Databricks, and Azure DevOps.
- Hands-on experience with Microsoft Fabric, including Data Factory, Lakehouse, Warehouse, OneLake, Power BI, and Notebooks.
- Strong SQL development and data modeling skills.
- Experience developing data pipelines using Python and/or PySpark.
- Understanding of star and snowflake data warehouse schemas.
- Experience working in Agile delivery environments.
- Strong stakeholder engagement and problem-solving skills.
- Experience with Snowflake, Databricks Lakehouse architecture, Microsoft Purview, streaming data architectures, or consulting environments is desirable.
- Experience integrating AI and generative AI capabilities, including Azure AI Services, Azure OpenAI, Microsoft Copilot Studio, or Microsoft Fabric AI capabilities, is advantageous.
- Knowledge of vector databases, Retrieval-Augmented Generation, and modern AI architectures is advantageous.
- Microsoft Azure certifications such as Azure Data Engineer Associate, Fabric Analytics Engineer Associate, or Azure Fundamentals are highly regarded.
Tech Stack
Categories
Data Engineering