6 days ago
Base Salary
$130k - $160k/yr
Responsibilities
- Design and implement enterprise-scale data platforms using Databricks Lakehouse architecture.
- Architect and develop scalable ETL/ELT pipelines for batch and real-time processing.
- Define data models, ingestion frameworks, transformation strategies, and data engineering standards.
- Implement Delta Lake, Unity Catalog, data governance, lineage, cataloging, security controls, and data quality frameworks.
- Build and optimize Spark and PySpark solutions for large-volume datasets.
- Lead cloud data platform modernization and migration initiatives across Azure, AWS, or GCP.
- Serve as technical lead for Databricks and data engineering programs, including architecture reviews, code reviews, and solution assessments.
- Establish coding standards, architecture guidelines, monitoring, observability, CI/CD, and Infrastructure-as-Code practices.
- Mentor Data Engineers, Technical Leads, and junior team members.
- Collaborate with architects, product owners, data scientists, business stakeholders, security teams, and compliance teams.
- Translate strategic business data requirements into scalable technical solutions.
- Drive delivery planning, risk management, resource coordination, and technical presentations to leadership.
- Manage database systems, performance tuning, security, and compliance standards.
Requirements
- 10-12 years of experience is stated in the job overview, with 12-15 years of overall IT experience also required and a qualifications section listing 15-18 years.
- At least 6 years of hands-on Databricks experience and 7 years of Spark/PySpark-based data engineering experience.
- Experience designing enterprise-scale Data Lake and Lakehouse solutions and leading large data transformation and modernization programs.
- Experience processing structured, semi-structured, and unstructured data and supporting analytics, AI/ML, and reporting ecosystems.
- Proven experience managing technical teams and driving architectural decisions.
- Bachelor's or master's degree in Computer Science, Engineering, Information Technology, or a related field; the qualifications section also references a four-year bachelor's or professional degree.
- Databricks Certified Data Engineer Professional, Azure Data Engineer Associate, AWS Data Analytics Specialty, Google Professional Data Engineer, TOGAF, or cloud architecture certifications are preferred.
- Preferred exposure includes MLOps, AI Engineering, Snowflake and Databricks integration, Microsoft Fabric, Data Mesh, Collibra, Alation, Informatica, Great Expectations, and Deequ.
Tech Stack
Amazon RedshiftApache KafkaApache SparkAWSAzureDatabricksDockerGitGitHub ActionsGoogle BigQueryGoogle Cloud PlatformInformaticaJenkinsKubernetesPythonScalaSnowflakeSQLTerraform
Categories
Data Engineering
About EXL
EXL provides data analytics, AI, and digital operations services for enterprises in insurance, healthcare, banking, and other regulated industries. It delivers consulting, managed services, and technology platforms to improve decision-making, risk management, and operational efficiency. Founded in 1999 and headquartered in New York, EXL is a public company listed on the NASDAQ (ticker: EXLS).