3 months ago
Mississauga, CanadaStaff+
Base Salary
$121k - $171k/yr
Responsibilities
- Lead the architecture, design, development, optimization, and operation of high-throughput batch and real-time streaming data pipelines using Apache Spark, Databricks, and Kafka.
- Serve as the senior subject-matter expert for Spark architecture, job optimization, SQL query tuning, distributed processing, Databricks performance engineering, and Spark execution analysis.
- Write, review, and approve Scala and Python code while enforcing coding, testing, deployment, monitoring, data quality, operational readiness, and production-support standards.
- Diagnose performance bottlenecks involving data skew, shuffling, partitioning, caching, joins, execution plans, and memory management.
- Manage and optimize Databricks workspaces, including Unity Catalog, Delta Live Tables, job clusters, security, governance, and monitoring.
- Implement and manage Apache Iceberg or Delta Lake table formats and enterprise lakehouse practices, including schema evolution, compaction, hidden partitioning, and medallion architecture.
- Define and enforce enterprise data-modeling and lakehouse standards across Bronze, Silver, and Gold layers.
- Lead, mentor, and develop a global team of data engineers through code reviews, architecture reviews, and technical design sessions.
- Manage technical priorities, delivery milestones, dependencies, risks, and collaboration with stakeholders, product owners, architects, and support teams.
- Assess and manage technology, operational, compliance, and control risks in accordance with applicable policies, laws, and regulations.
Requirements
- At least 6 years of relevant experience in application development or systems analysis.
- Expert-level development skills in Scala and Python.
- Advanced SQL development and query-optimization skills, including tuning slow-running queries and Spark jobs.
- Strong knowledge of data warehousing, lakehouse modeling, and medallion architecture.
- Required hands-on experience with Apache Iceberg or Delta Lake.
- Extensive hands-on experience with Apache Spark, Spark Structured Streaming, Databricks, Delta Live Tables, and Kafka.
- Deep knowledge of Spark internals, including the Catalyst Optimizer, Tungsten execution engine, memory management, execution plans, partitioning, and shuffle behavior.
- Experience architecting and operating multi-terabyte to petabyte-scale data platforms in AWS or Google Cloud Platform.
- Experience managing and implementing successful projects and leading or mentoring global or distributed data-engineering teams.
- Demonstrated leadership, project management, communication, stakeholder-management, problem-solving, and prioritization skills.
- Bachelor’s or university degree or equivalent experience required; a master’s degree is preferred.
- Subject-matter expertise in at least one area of applications development.
Benefits
- Full-time position.
- Primary location salary range is $120,800.00 to $170,800.00.
- Citi is an equal opportunity employer and provides consideration without regard to legally protected characteristics.
Tech Stack
Categories
Data Engineering
About Citi
Citi is a public financial-services company offering consumer and institutional banking, credit cards, wealth management, treasury and trade solutions, and capital-markets services. It serves individuals, corporations, financial institutions, and governments in more than 160 countries and jurisdictions, earning interest and fee income from lending, payments, trading, and advisory. Founded in 1812 and headquartered in New York, it trades on the NYSE under the ticker C.