20 hours ago
Mumbai, India or Chennai, IndiaSenior
Responsibilities
- Design, develop, and maintain scalable ETL/ELT data pipelines using Azure Data Factory, Azure Databricks, and Azure storage services.
- Ingest data from relational databases, APIs, and file-based systems into Azure data platforms.
- Implement transformations on HPCC using ECL and on Azure using Spark SQL and PySpark in Azure Databricks.
- Develop and optimize Azure Databricks notebooks and jobs and write efficient Spark jobs for large-scale processing.
- Design and implement data models using HPCC/ECL, Delta Lake, Parquet, and other columnar storage formats.
- Optimize ECL code, SQL, and Spark workloads for performance and scalability.
- Collaborate with business analysts, data scientists, and application teams to understand data requirements and promote data engineering best practices.
Requirements
- Strong proficiency in SQL and data transformation logic, with a solid understanding of data warehousing and ETL/ELT patterns.
- Hands-on experience with Azure Databricks, Spark SQL, and PySpark.
- Experience with Azure Data Factory, Azure Data Lake Storage Gen2, Azure SQL, and Synapse.
- Experience working with large-scale structured and semi-structured datasets.
- Knowledge of Agile development methodologies and the SDLC, including requirements gathering, design, implementation, testing, deployment, and maintenance.
- Solid understanding of software development principles, design patterns, and best practices.
- Proficiency in requirements gathering and documentation.
- Strong verbal and written communication and problem-solving skills.
- Knowledge of insurance terminology.
- Positive attitude and willingness to learn new processes and technologies.
Benefits
- Country-specific benefits are offered based on the employee's location.
- Accommodation and adjustment support is available throughout the hiring process.
About RELX
RELX provides information-based analytics, research content, and decision tools for scientists, legal and risk professionals, insurers, and governments. Through segments including Elsevier (STM), LexisNexis (legal and risk), and RX (exhibitions), it sells subscriptions, data services, and software platforms used in 180+ countries. Headquartered in London, it is a public company listed in London and Amsterdam.