Interview Bank

RAG in 14 Days: From Retrieval to Trustworthy Answers

D9 Hybrid Search and Reranking: Two-Path Retrieval, Reciprocal Rank Fusion, Then Re-Ranking the Top Results With a Cross-Encoder

• Why do hybrid retrieval systems usually use reciprocal rank fusion instead of normalizing both scores and adding them with weights? When does the weighted approach break down?混合检索为什么普遍用倒数排名融合，而不是把两路分数归一化之后加权相加？加权那条路在什么情况下会失控？

  Common in ChinaCommon overseasIntermediate#hybrid-search#rank-fusion

  How to reason about it · think before answering

  1. The hinge word is `scores`. Answering `RRF is simpler` is reciting a concept; the interviewer wants to hear that you know why the two scores are not comparable in the first place.
  2. Start with scale: BM25 is an unbounded sum of log terms, and on one index the top hit can range from 5 to 50 depending on the query; cosine is pinned between -1 and 1. Adding those two readings is meaningless.
  3. Then name the silent failure of normalization: dividing by the per-route maximum makes the denominator float with the query. For a question with no answer in the corpus, the vector route's best hit may score 0.09 and still normalize to a perfect 1.0, entering the fusion at full weight. You think you are comparing relevance; you are comparing `tallest among the short`.
  4. Then the maintenance cost of weights: a 1-to-0.6 ratio has to be tuned against an eval set, tuning two routes is a 2-D search, adding multi-query retrieval makes it 4-D or 5-D, and swapping the embedding model invalidates all of it. RRF has a single k, and the default of 60 rarely needs touching.
  5. Conclusion: rank is the only thing the two routes share. RRF throws the scores away on purpose so that an incomparable quantity cannot mislead it.
  6. Expected follow-up: what does k do? It flattens — the larger k is, the smaller the gap between the top few ranks, so `ranked well by both routes` outweighs `ranked first by one route`, which is exactly the cross-validation effect hybrid retrieval is after. A second follow-up on ties: you must fall back to sorting by document id, or ranks drift between runs and every eval number wobbles with them.

  分析过程 · 先想清楚再作答

  1. 这题的题眼在「分数」两个字。只答「RRF 更简单」是背概念，面试官想听的是你知道分数为什么不可比。
  2. 先给量纲差异：BM25 是一堆对数项累加，没有上界，同一套索引里不同查询的第一名可以从 5 分到 50 分；余弦被钉死在负一到正一。两个读数相加没有意义。
  3. 再点出归一化的静默失败：除以本路最高分之后，分母随查询浮动。一个语料里根本没有答案的问题，向量那一路最高分只有 0.09，归一化之后照样是满分 1.0 带权重进融合——你以为在比相关性，其实在比「本路矮子里有多高」。
  4. 然后是权重的维护成本：1 比 0.6 这个配比要靠跑评估调出来，两路是二维搜索，加上多路查询就是四维五维，而且换一个 embedding 模型全部作废。RRF 只有一个 k，而且 60 这个默认值几乎不用动。
  5. 结论：名次是两路唯一可比的东西。RRF 主动扔掉分数，是为了不被不可比的量误导。
  6. 可预期的追问：那 k 是干什么的？答 k 是压平器——k 越大，头几名之间的差距越小，于是「两路都排进前列」比「一路排第一」更有分量，这正是混合检索想要的交叉验证效果。再追问同分怎么办，答必须按文档 id 兜底排序，否则跨次运行名次会飘、评估数字跟着抖。

  Key points

  • BM25 is unbounded, cosine is bounded; the two scales are not comparable, so adding them is meaningless.
  • Per-route max normalization has a denominator that floats with the query, so the least relevant hit of an unanswerable query still normalizes to 1.0.
  • Weights must be tuned against an eval set, the search is high-dimensional once you add routes, and swapping models invalidates it; RRF has a single constant k.
  • RRF consumes only the ordered id list from each route, because rank is the one thing the routes share.
  • Larger k rewards `ranked well by both routes`; ties must fall back to document id so results are reproducible.

  答题要点

  • BM25 无上界、余弦有界，两个量纲不可比，直接相加没有意义。
  • 按本路最高分归一化的分母随查询浮动，无答案的查询里最不相干的结果也能拿到满分。
  • 权重要跑评估调，路数一多就是高维搜索，换模型还得重来；RRF 只有一个常数 k。
  • RRF 只吃每一路的有序 id 列表，名次是两路唯一可比的东西。
  • k 越大越奖励「两路都排进前列」；同分必须按 id 兜底排序才可复现。