Staff Software Engineer- Search Quality
Databricks8 months ago
Bengaluru, IndiaStaff+
Responsibilities
- Own the quality of search and retrieval results for AI agents and human users.
- Optimize hybrid keyword and semantic vector retrieval for precision, recall, intent, and readability.
- Fine-tune ranking models and build relevance scoring and guardrails for grounded AI responses.
- Develop frameworks to measure search quality using human and LLM-based evaluation.
- Improve discovery across heterogeneous enterprise data sources.
Requirements
- Expertise in information retrieval and familiarity with Lucene or Elasticsearch, embeddings, and ranking algorithms.
- Experience or strong interest in building evaluation frameworks using human-in-the-loop and LLM-based methods.
- Deep understanding of relevance metrics including nDCG, MRR, and Precision@K.
- Interest in applying traditional information-retrieval techniques to retrieval-augmented generation.
- Ability to work across structured SQL tables, unstructured documents, and real-time business metrics.
Benefits
- Comprehensive employee benefits and perks, with details varying by region.
Tech Stack
Categories
About Databricks
Databricks builds a cloud-based data and AI platform centered on the lakehouse architecture, combining data engineering, analytics, and machine learning with Apache Spark, Delta Lake, and MLflow. It sells subscriptions and cloud services to enterprises that need to unify data pipelines and develop large-scale AI and analytics. Founded in 2013 by the creators of Apache Spark and headquartered in San Francisco, the company is privately held and serves organizations across many industries.