Senior Staff Software Engineer- Search Quality
Databricks8 months ago
Bengaluru, IndiaStaff+
Responsibilities
- Own the quality of search results for AI agents and human users.
- Optimize retrieval across keyword-based and semantic vector search.
- Fine-tune ranking models for human readability and LLM-ready context.
- Build guardrails and relevance scoring to reduce hallucinations and keep AI grounded in source data.
- Connect structured SQL tables, unstructured documents, and real-time business metrics.
- Develop evaluation frameworks to measure search relevance.
Requirements
- Expertise in information retrieval, including Lucene or Elasticsearch, embeddings, and ranking algorithms.
- Experience with relevance evaluation frameworks, including human-in-the-loop and LLM-based approaches.
- Strong understanding of search relevance metrics such as nDCG, MRR, and Precision@K.
- Interest in applying traditional information retrieval techniques to Retrieval-Augmented Generation systems.
Benefits
- Comprehensive employee benefits and perks are offered, with region-specific details available through Databricks' benefits portal.
Tech Stack
Categories
About Databricks
Databricks builds a cloud-based data and AI platform centered on the lakehouse architecture, combining data engineering, analytics, and machine learning with Apache Spark, Delta Lake, and MLflow. It sells subscriptions and cloud services to enterprises that need to unify data pipelines and develop large-scale AI and analytics. Founded in 2013 by the creators of Apache Spark and headquartered in San Francisco, the company is privately held and serves organizations across many industries.