10 hours ago
Responsibilities
- Define the enterprise target-state data architecture on Databricks across ingestion, medallion layering, serving, ML, source systems, legacy warehouses, loan servicing platforms, and vendor feeds.
- Run discovery with business and technology stakeholders and produce solution designs, reference architectures, and delivery roadmaps.
- Design marketing data architecture for identity resolution, segmentation, campaign measurement, attribution, and compliant data sharing.
- Design collections data architecture for near-real-time delinquency and contact data, vendor exchanges, contact preferences, and regulatory restrictions.
- Make cross-cutting decisions about batch versus streaming, build versus buy, workspace topology, network and multi-account design, and disaster recovery.
- Design security and compliance architecture with InfoSec, including Unity Catalog access models, PII/NPI protection, encryption, audit logging, and lineage.
- Plan migration and cutover strategies for legacy Hadoop, Hive, EMR, and warehouse workloads while mitigating delivery risks.
- Build DBU forecasts, cost and capacity models, FinOps guardrails, and business cases for platform investments.
- Chair the architecture review board and maintain standards, reference patterns, and architecture decision records.
- Manage Databricks and cloud-provider relationships and lead vendor evaluations and proofs of concept.
- Provide organizational data engineering oversight, lead high-impact initiatives, ensure alignment with organizational goals, and provide executive guidance.
Requirements
- 12+ years of experience in data engineering or architecture, including 5+ years on the Databricks lakehouse platform.
- Strong experience with Spark, Delta Lake, SQL, and data modeling including medallion and dimensional modeling.
- AWS architecture experience covering networking, IAM, encryption, and multi-account design.
- Experience designing secure platforms for PII/NPI, including Unity Catalog access models and data lineage.
- Experience planning migrations from legacy Hadoop, Hive, and warehouse systems.
- Ability to design both batch and streaming systems using Kafka and Structured Streaming.
- Experience with disaster recovery and high-availability design.
- Experience with FinOps, Databricks cost and capacity modeling, and business-case development.
- Strong stakeholder management, vendor management, and architecture documentation skills.
- Graduate-level education in Computer Science, Data Science, or a related field.
- Preferred experience in banking, lending, regulated financial services, marketing data architecture, or collections data architecture.
- Preferred experience with ML/MLOps architecture on Databricks, including feature stores and MLflow.
- Databricks and AWS Solution Architect certifications are preferred.
- The qualifications section also states 6–8 years of experience in data engineering or a related field.
Tech Stack
Categories
Data EngineeringSolutions Engineering
About EXL
EXL provides data analytics, AI, and digital operations services for enterprises in insurance, healthcare, banking, and other regulated industries. It delivers consulting, managed services, and technology platforms to improve decision-making, risk management, and operational efficiency. Founded in 1999 and headquartered in New York, EXL is a public company listed on the NASDAQ (ticker: EXLS).