Interview Bank

From Frontend Engineer to Agent Engineer in 30 Days

D24 RAG, Level Up: Hybrid Search, Reranking, Citations, Recall Evaluation

• How do you merge two retrieval rankings, and why not just take a weighted sum of the scores?两路检索结果怎么合并？为什么不能直接加权求和？

  Common in ChinaCommon overseasIntermediate#rag#rrf#ranking

  How to reason about it · think before answering

  1. The second half is the real question. Anyone can say 'RRF'; explaining why weighted sums fail is what separates people who have looked at the score distributions.
  2. Decompose it: are the two scores even the same unit? Cosine similarity is bounded in 0 to 1 and tightly clustered — candidates often differ by 0.02. BM25 is unbounded and a few rare-term hits reach 12. Adding them lets the larger-magnitude channel decide everything; the weight only tunes how much it dominates.
  3. Worse, it is unstable. Weights tuned on one corpus drift on the next, so you re-tune forever.
  4. Conclusion: fuse ranks, not scores. RRF maps each rank to 1/(k + rank) and sums, with k = 60. Ranks are unitless and need no calibration. k flattens the head of the list so that 'top-ranked in both channels' beats 'first in one channel' — consensus over single-source confidence.
  5. A hand-checkable example helps: rankings [a,b,c] and [c,d,a] give a = 1/61 + 1/63 ≈ 0.0323, while a raw score sum promotes c on the strength of its BM25 12.
  6. Expected follow-up: what about ties? You must break them explicitly, e.g. by id. Otherwise ordering depends on hash-map iteration order and differs across languages and runs, which makes your evaluation numbers irreproducible. Mentioning this signals you actually ran it more than once.

  分析过程 · 先想清楚再作答

  1. 题眼在后半句。前半句答「RRF」谁都会，后半句「为什么不能加权求和」才是筛人的地方——它考的是你有没有真的看过两路分数的分布。
  2. 怎么拆：先问自己两个分数是不是同一个量纲。余弦相似度有界（0 到 1）且分布密集，同一批候选常常只差 0.02；BM25 无上界，命中几个稀有词就能到 12 分。**不同量纲的数相加，等于让量纲大的那一路单方面决定结果**，权重只是在调「它说了算的程度」。
  3. 更麻烦的是它不稳定：权重在这批语料上调好了，换一批语料分布就变了，得重调。这是一个永远还不完的技术债。
  4. 结论：改用名次。RRF 把每一路的名次折算成 `1/(k + rank)` 再相加，k 取 60。名次是无量纲的，不需要任何标定。k 的作用是压平头部差距，让「两路都进前列」压过「一路排第一」——共识优先于单点自信。
  5. 一个能当场手算的例子很加分：两路排名 [a,b,c] 与 [c,d,a]，a 得 1/61 + 1/63 ≈ 0.0323；而分数直接相加的版本会把 BM25 里 12 分的 c 顶到第一。
  6. 可预期的追问：同分了怎么办？必须显式定序（比如按 id），否则结果取决于哈希表遍历顺序，同一份输入在不同语言、不同运行里给出不同排序——评估集量出来的数字也就不可复现了。这一条答出来会非常加分，因为它说明你真的跑过多次。

  Key points

  • Use RRF: map each channel's rank to 1/(k + rank) and sum, with k = 60.
  • Weighted sums fail because the scores are different units — bounded, tightly clustered cosine versus unbounded BM25, so BM25 decides the outcome.
  • Weights also do not transfer: tuned on one corpus, they drift on the next.
  • Ranks are unitless and need no calibration; k flattens the head so cross-channel consensus outweighs single-channel confidence.
  • Break ties explicitly (by id) or ordering depends on hash iteration order and your evaluation numbers stop being reproducible.

  答题要点

  • 用 RRF：每一路的名次折算成 1/(k + rank) 再相加，k 取 60。
  • 不能加权求和是因为两个分数量纲不同——余弦有界密集、BM25 无上界，相加等于让 BM25 单方面决定结果。
  • 而且权重不可迁移：这批语料调好，换一批就得重调，是还不完的债。
  • 名次是无量纲的，不需要标定；k 压平头部差距，让两路共识压过单路自信。
  • 同分必须显式定序（按 id），否则结果依赖哈希表遍历顺序，评估数字不可复现。