Kyndryl

Senior Data Engineer – Databricks, dbt & GenAI

Kyndryl
Apply
23 hours ago
Chennai, India or Bengaluru, IndiaSenior

Responsibilities

  • Design and build scalable data pipelines for Generative AI, advanced analytics, and enterprise data initiatives.
  • Develop and optimize RAG frameworks, vectorized data processing pipelines, semantic data layers, knowledge frameworks, and contextual datasets.
  • Develop and manage dbt models, macros, tests, documentation, source management, and deployment pipelines.
  • Use Databricks Lakehouse capabilities including Delta Lake, Databricks SQL, Unity Catalog, and Workflows.
  • Implement data quality controls, governance, validation, security, lineage, and compliance processes.
  • Collaborate with AI Engineers, Data Scientists, Architects, Cloud Engineers, business stakeholders, and global teams.
  • Build, deploy, monitor, troubleshoot, and optimize CI/CD pipelines and cloud-based data platforms.
  • Drive best practices in data modeling, version control, testing, documentation, security, and cloud-native development.
  • Mentor junior engineers through code reviews, pair programming, technical coaching, and engineering best practices.

Requirements

  • At least 7 years of hands-on Data Engineering experience designing, deploying, and supporting enterprise-scale production data solutions.
  • Expert proficiency in dbt Core, including project structure, models, macros, testing, documentation, source management, configuration, CI/CD integration, and troubleshooting.
  • Strong hands-on experience with the Databricks Lakehouse Platform, including Databricks SQL, Unity Catalog, Delta Lake, Workflows, and performance optimization.
  • Advanced SQL skills covering complex transformations, query optimization, dimensional modeling, partitioning, data warehousing, and large-scale analytical workloads.
  • Experience designing dimensional models including SCD Type 1 and Type 2, enterprise reporting layers, and scalable data products.
  • Proficiency in Python for data engineering, automation, orchestration, and ingestion frameworks.
  • Experience building ingestion pipelines from REST APIs, SFTP, relational databases, cloud storage, and large-scale file datasets.
  • Strong understanding of LLMs, RAG, embeddings, vector databases, semantic layers, and AI-ready data architectures.
  • Experience preparing, governing, and optimizing enterprise data for AI, machine learning, and Generative AI use cases.
  • Experience with enterprise data governance, security, lineage, quality frameworks, metadata management, data cataloging, and compliance.
  • Strong understanding of cloud services including ADLS, GCS, Azure Key Vault, GCP Secret Manager, compute services, containers, and serverless architectures.
  • Experience supporting CI/CD pipelines using GitHub Actions and modern DevOps practices.
  • Strong troubleshooting, monitoring, performance tuning, and operational support experience in cloud-based data platforms.
  • Excellent communication, collaboration, and stakeholder management skills.
  • Preferred qualifications include Azure and Google Cloud Platform experience, multi-cloud environments, dlt or similar ingestion and orchestration tools, Terraform, Power BI integration, advanced Python development, and experience mentoring junior engineers.

Benefits

  • Hybrid-friendly work culture supporting employee well-being.
  • Be Well programs supporting financial, mental, physical, and social health.
  • Personalized development goals, continuous feedback, coaching, and career-path tools.
  • Learning opportunities and certifications with Microsoft, Google, and Amazon.
  • Collaborative culture emphasizing inclusion, empathy, continuous learning, and shared success.

Tech Stack

Categories

Data Engineering
Kyndryl

About Kyndryl

10,000+ employees

Kyndryl is a public IT services company that designs, runs, and modernizes mission-critical infrastructure for large enterprises and governments. Spun off from IBM in 2021, it is headquartered in New York City and trades on the NYSE (ticker: KD). The company provides managed infrastructure, cloud migration and operations across AWS, Azure, and Google Cloud, plus mainframe, network, security, data, and digital workplace services.

Contact me