2 months ago
Bengaluru, IndiaSenior
Responsibilities
- Deconstruct and migrate Apache Spark and Hadoop MapReduce workflows into production-ready AWS and Databricks pipelines.
- Build and deploy intelligent AI agents and workflow automation tools using Anthropic Claude and other LLM APIs.
- Develop AI data pipelines for unstructured data ingestion, real-time feature tokenization, and vector-database metadata tagging.
- Monitor, maintain, and troubleshoot Hadoop workloads, including resolving bottlenecks and out-of-memory errors during migration.
- Support Tableau, Looker, and other BI users through functional enhancements, bug fixes, extract optimization, and dashboard-latency troubleshooting.
- Use Databricks and Delta Lake features such as ACID transactions, Z-Ordering, and caching to improve pipeline performance, reliability, and cost efficiency.
- Implement Databricks AI tools including Databricks Vector Search, Mosaic AI, and Lakeflow.
Requirements
- 5–8 years of experience in data engineering or a closely related field.
- Hands-on experience building functional AI agents, integrating LLM APIs such as Anthropic Claude, and using orchestration frameworks such as LangChain or the Databricks Mosaic AI Agent Framework.
- Strong Apache Spark experience and understanding of distributed storage and computing concepts including partitioning, shuffling, caching, and broadcast joins.
- Strong proficiency in Python with PySpark or Scala, plus intermediate-to-advanced SQL skills including window functions, query tuning, and complex joins.
- Direct experience or deep theoretical knowledge of AWS, including S3 and IAM, and Databricks environments.
- Practical experience with BI tools such as Tableau, Looker, or Power BI, including debugging calculated fields, modifying parameters, and troubleshooting slow reports.
Tech Stack
Categories
AI ApplicationsData Engineering
About EXL
EXL provides data analytics, AI, and digital operations services for enterprises in insurance, healthcare, banking, and other regulated industries. It delivers consulting, managed services, and technology platforms to improve decision-making, risk management, and operational efficiency. Founded in 1999 and headquartered in New York, EXL is a public company listed on the NASDAQ (ticker: EXLS).