23 hours ago
Chennai, India or Bengaluru, IndiaSenior
Responsibilities
- Design and build scalable data pipelines for Generative AI, advanced analytics, and enterprise data initiatives.
- Develop and optimize RAG frameworks, vectorized data processing pipelines, semantic data layers, knowledge frameworks, and contextual datasets.
- Develop and manage dbt models, macros, tests, documentation, source management, and deployment pipelines.
- Use Databricks Lakehouse capabilities including Delta Lake, Databricks SQL, Unity Catalog, and Workflows.
- Implement data quality controls, governance, validation, security, lineage, and compliance processes.
- Collaborate with AI Engineers, Data Scientists, Architects, Cloud Engineers, business stakeholders, and global teams.
- Build, deploy, monitor, troubleshoot, and optimize CI/CD pipelines and cloud-based data platforms.
- Drive best practices in data modeling, version control, testing, documentation, security, and cloud-native development.
- Mentor junior engineers through code reviews, pair programming, technical coaching, and engineering best practices.
Requirements
- At least 7 years of hands-on Data Engineering experience designing, deploying, and supporting enterprise-scale production data solutions.
- Expert proficiency in dbt Core, including project structure, models, macros, testing, documentation, source management, configuration, CI/CD integration, and troubleshooting.
- Strong hands-on experience with the Databricks Lakehouse Platform, including Databricks SQL, Unity Catalog, Delta Lake, Workflows, and performance optimization.
- Advanced SQL skills covering complex transformations, query optimization, dimensional modeling, partitioning, data warehousing, and large-scale analytical workloads.
- Experience designing dimensional models including SCD Type 1 and Type 2, enterprise reporting layers, and scalable data products.
- Proficiency in Python for data engineering, automation, orchestration, and ingestion frameworks.
- Experience building ingestion pipelines from REST APIs, SFTP, relational databases, cloud storage, and large-scale file datasets.
- Strong understanding of LLMs, RAG, embeddings, vector databases, semantic layers, and AI-ready data architectures.
- Experience preparing, governing, and optimizing enterprise data for AI, machine learning, and Generative AI use cases.
- Experience with enterprise data governance, security, lineage, quality frameworks, metadata management, data cataloging, and compliance.
- Strong understanding of cloud services including ADLS, GCS, Azure Key Vault, GCP Secret Manager, compute services, containers, and serverless architectures.
- Experience supporting CI/CD pipelines using GitHub Actions and modern DevOps practices.
- Strong troubleshooting, monitoring, performance tuning, and operational support experience in cloud-based data platforms.
- Excellent communication, collaboration, and stakeholder management skills.
- Preferred qualifications include Azure and Google Cloud Platform experience, multi-cloud environments, dlt or similar ingestion and orchestration tools, Terraform, Power BI integration, advanced Python development, and experience mentoring junior engineers.
Benefits
- Hybrid-friendly work culture supporting employee well-being.
- Be Well programs supporting financial, mental, physical, and social health.
- Personalized development goals, continuous feedback, coaching, and career-path tools.
- Learning opportunities and certifications with Microsoft, Google, and Amazon.
- Collaborative culture emphasizing inclusion, empathy, continuous learning, and shared success.
Tech Stack
Categories
Data Engineering
About Kyndryl
Kyndryl is a public IT services company that designs, runs, and modernizes mission-critical infrastructure for large enterprises and governments. Spun off from IBM in 2021, it is headquartered in New York City and trades on the NYSE (ticker: KD). The company provides managed infrastructure, cloud migration and operations across AWS, Azure, and Google Cloud, plus mainframe, network, security, data, and digital workplace services.
