23 hours ago
Chennai, India or Bengaluru, IndiaSenior
Responsibilities
- Design and build scalable data pipelines supporting Generative AI, advanced analytics, and enterprise data initiatives.
- Develop and optimize RAG frameworks and vectorized data processing pipelines for AI-driven and agentic workflows.
- Build semantic data layers, knowledge frameworks, contextual datasets, and governed reusable data products.
- Develop and manage dbt models, macros, tests, documentation, source management, and deployment pipelines.
- Use Databricks Lakehouse capabilities, including Delta Lake, Databricks SQL, Unity Catalog, and Workflows, to deliver scalable data solutions.
- Implement automated data quality, governance, validation, security, lineage, and compliance controls.
- Collaborate with AI Engineers, Data Scientists, Architects, Cloud Engineers, business stakeholders, and global teams.
- Build, deploy, and maintain CI/CD pipelines for data engineering solutions.
- Monitor, troubleshoot, and optimize data platform performance, reliability, scalability, and operational efficiency.
- Drive best practices in data modeling, version control, testing, documentation, security, and cloud-native development.
- Mentor junior engineers through code reviews, pair programming, technical coaching, and engineering best practices.
Requirements
- At least 7 years of hands-on data engineering experience designing, deploying, and supporting enterprise-scale production data solutions.
- Expert proficiency in dbt Core, including project structuring, model development, macros, testing, documentation, source management, environment configuration, CI/CD integration, and troubleshooting.
- Strong hands-on experience with the Databricks Lakehouse Platform, including Databricks SQL, Unity Catalog, Delta Lake, Databricks Workflows, and performance optimization.
- Advanced SQL proficiency covering complex transformations, query optimization, dimensional modeling, partitioning, data warehousing, and large-scale analytical workloads.
- Experience designing dimensional models, including SCD Type 1 and Type 2, enterprise reporting layers, and scalable data products.
- Proficiency in Python for data engineering, automation, orchestration, and ingestion frameworks, with preferred experience in OOP, decorators, multithreading, and optimization.
- Experience building ingestion pipelines from REST APIs, SFTP, relational databases, cloud storage, and large-scale file-based datasets.
- Strong understanding of Generative AI concepts, including LLMs, RAG, embeddings, vector databases, semantic layers, and AI-ready data architectures.
- Experience preparing, governing, and optimizing enterprise data for AI, machine learning, and GenAI use cases.
- Experience implementing enterprise data governance, security, lineage, and quality frameworks using Unity Catalog and dbt testing.
- Understanding of cloud services including object storage, secret management, compute, containers, and serverless architectures.
- Experience establishing and supporting CI/CD pipelines using GitHub Actions and modern DevOps practices.
- Strong troubleshooting, performance tuning, monitoring, and operational support experience in cloud-based data platforms.
- Excellent communication, collaboration, and stakeholder management skills.
- Preferred experience includes Azure and Google Cloud Platform, dlt or similar ingestion and orchestration frameworks, metadata management, data cataloging, Agentic AI, GenAI, machine learning, advanced analytics, Terraform, Power BI integration, and mentoring junior engineers.
Benefits
- Hybrid-friendly work culture with programs supporting financial, mental, physical, and social well-being.
- Personalized development goals, continuous feedback, coaching, and career-path tools.
- Access to learning opportunities and certifications with Microsoft, Google, and Amazon.
- Mentorship, hands-on experiences, and opportunities to grow toward roles such as Lead Data Engineer, Data Architect, AI Platform Engineer, GenAI Solutions Architect, or Data Engineering Manager.
Tech Stack
Categories
Data Engineering
About Kyndryl
Kyndryl is a public IT services company that designs, runs, and modernizes mission-critical infrastructure for large enterprises and governments. Spun off from IBM in 2021, it is headquartered in New York City and trades on the NYSE (ticker: KD). The company provides managed infrastructure, cloud migration and operations across AWS, Azure, and Google Cloud, plus mainframe, network, security, data, and digital workplace services.
