6 days ago
Base Salary
$100k - $135k/yr
Responsibilities
- Design and implement enterprise-scale Databricks Lakehouse data platforms and scalable batch and real-time ETL/ELT pipelines.
- Define data models, ingestion frameworks, transformation strategies, governance, lineage, cataloging, and security controls.
- Build and optimize Spark, PySpark, Structured Streaming, and Delta Lake solutions for large-volume datasets.
- Lead cloud data platform modernization, migration, CI/CD, infrastructure-as-code, monitoring, observability, and operational excellence initiatives.
- Serve as technical lead for Databricks and data engineering programs, establishing standards and conducting architecture, code, and solution reviews.
- Mentor data engineers, technical leads, and junior team members while coordinating with architects, product owners, data scientists, security teams, and business stakeholders.
- Translate business requirements into scalable technical solutions and drive delivery planning, risk management, resource coordination, and executive presentations.
- Implement data quality monitoring and ensure compliance with organizational and regulatory data protection requirements.
Requirements
- 12-15 years of overall IT experience.
- 8-10 years of experience in data engineering, big data, and cloud data platforms.
- At least 6 years of hands-on Databricks experience.
- At least 7 years of Spark/PySpark-based data engineering experience.
- Experience designing enterprise-scale Data Lake and Lakehouse solutions and leading large data transformation and modernization programs.
- Experience processing structured, semi-structured, and unstructured data and supporting analytics, AI/ML, and reporting ecosystems.
- Proven experience managing technical teams and driving architectural decisions.
- Bachelor's or master's degree in Computer Science, Engineering, Information Technology, or a related field.
- Databricks Certified Data Engineer Professional, Azure Data Engineer Associate, AWS Data Analytics Specialty, Google Professional Data Engineer, TOGAF, or cloud architecture certifications are preferred.
- Experience with MLOps, AI engineering, Snowflake and Databricks integration, Microsoft Fabric, Data Mesh, Collibra, Alation, Informatica, Great Expectations, or Deequ is beneficial.
Benefits
- Full-time employment.
- Location: US.
Tech Stack
Amazon RedshiftApache KafkaApache SparkAWSAzureDatabricksDockerGitGitHub ActionsGoogle BigQueryGoogle Cloud PlatformInformaticaJenkinsKubernetesPythonScalaSnowflakeSQLTerraform
Categories
Data Engineering
About EXL
EXL provides data analytics, AI, and digital operations services for enterprises in insurance, healthcare, banking, and other regulated industries. It delivers consulting, managed services, and technology platforms to improve decision-making, risk management, and operational efficiency. Founded in 1999 and headquartered in New York, EXL is a public company listed on the NASDAQ (ticker: EXLS).