Staff Software Development Engineer (Data Engineer)

3 months ago

Bengaluru, IndiaStaff+

Responsibilities

Lead the design and development of hybrid vector and graph retrieval architectures.
Architect data pipelines for ingestion, embedding, and indexing of massive multimodal datasets.
Prototype advanced retrieval techniques including reranking, LLM graph tooling, and metadata filtering.
Design schemas for knowledge graphs and ensure high-performance relationship mapping and ontological integrity.
Build data validation and drift-detection systems for embedding quality and vector-space health.
Implement persistent, observable, low-latency memory systems for AI agents.
Unify disparate data sources into coherent, searchable knowledge bases and establish data organization standards.
Collaborate with AI Research and Product teams to evaluate and integrate emerging database and retrieval technologies.

7+ years of experience in data engineering or backend systems focused on high-performance data retrieval and storage.
Expert proficiency in Python, Java, or Go and strong knowledge of distributed-system design patterns.
Deep understanding of vector-database indexing strategies including HNSW, IVFFlat, and PQ, along with cosine, Euclidean, and dot-product distance metrics.
Experience with Pinecone, Milvus, Weaviate, or Qdrant.
Strong experience with graph databases such as Neo4j, AWS Neptune, or ArangoDB and query languages including Cypher or Gremlin.
Experience building semantic layers, ontologies, taxonomies, and other data models.
Hands-on experience with LangChain, LlamaIndex, and embedding models from OpenAI, HuggingFace, or Cohere.
Proficiency with Spark, Flink, or Kafka for large-scale processing, real-time indexing, and ETL.
Understanding of information-retrieval concepts including BM25, TF-IDF, and reciprocal rank fusion.
Experience with AWS, GCP, or Azure and Kubernetes.
Bachelor's or master's degree in computer science or a related field, with 7–9 years of professional experience preferred.

501-1,000 employees

We’ve moved! 🌟 Follow us on @JioHotstar for all the latest stories, updates & more.