
Data Engineer
Datamatics Technologies2 days ago
Chennai, IndiaMid Level
Responsibilities
- Build batch, incremental, CDC, and streaming ingestion pipelines from source systems into Bronze layers.
- Develop Silver- and Gold-layer transformations, cleansing, conformance, enrichment, business-rule logic, dimensional models, SCDs, surrogate keys, and data marts.
- Integrate data-quality checks, validation, quarantine, and reconciliation into pipelines.
- Publish metadata and lineage to Microsoft Purview.
- Implement scheduling, dependency management, parameterization, error handling, alerting, automated tests, and environment promotion.
- Optimize partitioning, Delta/Parquet file formats, and compute usage for performance and cost efficiency.
- Apply access controls, masking, encryption, and secure credential management in compliance with UAE PDPL, the Dubai Data Law, and DESC ISR.
- Support production operations and incident resolution, maintain runbooks, and coach counterparts during handover.
Requirements
- 4+ years of data engineering experience, preferably in UAE government or regulated-sector environments.
- Advanced SQL, Python, and PySpark/Spark skills.
- Hands-on experience with Azure Data Factory, Microsoft Fabric, Databricks, or Synapse.
- Strong understanding of lakehouse/medallion architecture, Delta/Parquet formats, dimensional modeling, SCDs, and surrogate keys.
- Experience with CI/CD, Azure DevOps or GitHub, source control, and DataOps practices.
- Familiarity with Microsoft Purview metadata and lineage and data-quality tooling.
- Awareness of UAE PDPL, the Dubai Data Law, and DESC ISR.
- Excellent English communication and documentation skills.
- Arabic proficiency is advantageous.
Benefits
- UAE-based assignment with travel required.
- Six-month assignment duration.
Tech Stack
Categories
Data Engineering
About Datamatics Technologies
Datamatics Technologies is a London-headquartered, privately held IT services firm founded in 2016 that delivers onsite and offshore data analytics and data science solutions. It provides resource augmentation and managed services—on time-and-materials or fixed-price terms—covering data warehousing, BI, ETL, information security, and cloud platform engineering with emphasis on Oracle Cloud. The company serves telecom, finance, government, and manufacturing clients, primarily across the Middle East and Africa.