3 days ago
Belfast, United KingdomStaff+
Responsibilities
- Build and optimize large-scale Scala Spark data pipelines on Databricks for financial data reconciliation.
- Modernize legacy Cloudera Hadoop pipelines into cloud-native Databricks and AWS architectures using Delta Lake and Databricks Workflows.
- Analyze Spark performance bottlenecks and simplify complex jobs into efficient, standardized patterns.
- Translate architectural decisions into technical implementations, data models, and reusable pipeline components.
- Collaborate with architects, platform teams, DevOps, cloud infrastructure teams, and business stakeholders.
- Develop and maintain unit, integration, and data-validation tests for reconciliation pipelines.
- Contribute to shared engineering frameworks, reusable libraries, standards, and best practices.
- Support the migration and continuous improvement of a business-critical reconciliation platform.
Requirements
- Experience with Scala Spark, PySpark, or Java Spark and comfort working across multiple Spark language APIs.
- Familiarity with AWS services including S3, Glue, IAM, CloudWatch, or Step Functions in a data engineering context.
- Experience or exposure to data reconciliation, financial data, reconciliation logic, data validation, and accuracy requirements at scale.
- Experience with CI/CD pipelines, automated testing, deployment pipelines, DevOps practices, and version control.
- Knowledge of data modeling and pipeline orchestration patterns beyond Databricks Workflows, such as Apache Airflow.
- Experience with large-scale Hadoop-to-cloud migrations or legacy platform modernization projects.
- Familiarity with streaming data processing such as Apache Kafka and Spark Structured Streaming alongside batch workloads.
- Experience contributing to shared engineering frameworks or reusable libraries in a platform team setting.
Benefits
- Hybrid working model with up to 2 days working from home per week.
- 27 days of annual leave plus bank holidays.
- Discretionary annual performance-related bonus.
- Private medical care and life insurance.
- Employee Assistance Program.
- Pension plan.
- Paid parental leave.
- Employee, family, and friends discounts.
- Access to learning and development resources.
- Business-casual workplace and full-time employment.
Tech Stack
Categories
Data Engineering
About Citi
Citi is a public financial-services company offering consumer and institutional banking, credit cards, wealth management, treasury and trade solutions, and capital-markets services. It serves individuals, corporations, financial institutions, and governments in more than 160 countries and jurisdictions, earning interest and fee income from lending, payments, trading, and advisory. Founded in 1812 and headquartered in New York, it trades on the NYSE under the ticker C.