2 days ago
Bengaluru, India or Hyderābād, IndiaStaff+
Responsibilities
- Architect and scale RAG pipelines that transform unstructured IT logs and documentation into optimized vector embeddings.
- Maintain the health and performance of vector databases such as Pinecone, Milvus, and Weaviate for sub-second retrieval.
- Build knowledge graphs and semantic layers that provide contextual information for AI agents.
- Create automated data guardrails to detect noise, bias, and personally identifiable information before data reaches models.
- Identify and resolve data-quality issues at their source.
- Build, deploy, and maintain data-infrastructure CI/CD pipelines.
- Develop metadata-driven ELT frameworks, reusable pipeline templates, data-quality frameworks, reconciliation utilities, and automated testing.
- Optimize partitioning, incremental processing, schema evolution, performance, and large-volume data processing.
- Apply healthcare data practices involving patient identity, PHI, lineage, effective dating, and audit requirements.
Requirements
- At least 7 years of experience in data engineering or data warehousing.
- At least 4 years of hands-on experience with Databricks, Spark, and Snowflake.
- Advanced hands-on experience with Databricks, Python, SQL, Spark, PySpark, Delta Lake, and Snowflake.
- Strong experience with Databricks Workflows/Jobs, notebooks, Unity Catalog, Git, and CI/CD.
- Knowledge of dimensional and normalized data modeling.
- Strong troubleshooting, debugging, and root-cause-analysis skills.
- Preferred: experience building GenAI or RAG applications on Databricks, including agentic AI, LLMs, prompt engineering, embeddings, and vector search.
- Preferred: experience with MLflow, AI/ML lifecycle practices, Azure OpenAI or equivalent enterprise LLM platforms, and agent frameworks.
- Preferred: experience taking AI or GenAI solutions from proof of concept to production, including monitoring, security, and governance.
- Preferred: healthcare domain experience and familiarity with PHI-sensitive datasets.
Benefits
- Hybrid-friendly work culture.
- Professional certifications and qualifications, including learning opportunities involving Microsoft, Google, and Amazon.
- Be Well programs supporting financial, mental, physical, and social health.
- Personalized development goals, career-path tools, coaching, continuous feedback, and hands-on learning experiences.
Tech Stack
Categories
Data Engineering
About Kyndryl
Kyndryl is a public IT services company that designs, runs, and modernizes mission-critical infrastructure for large enterprises and governments. Spun off from IBM in 2021, it is headquartered in New York City and trades on the NYSE (ticker: KD). The company provides managed infrastructure, cloud migration and operations across AWS, Azure, and Google Cloud, plus mainframe, network, security, data, and digital workplace services.
