Hybrid Search and Reranking: Two-Path Retrieval, Reciprocal Rank Fusion, Then Re-Ranking the Top Results With a Cross-Encoder

Interview questions

• Why do hybrid retrieval systems usually use reciprocal rank fusion instead of normalizing both scores and adding them with weights? When does the weighted approach break down?混合检索为什么普遍用倒数排名融合，而不是把两路分数归一化之后加权相加？加权那条路在什么情况下会失控？

  Common in ChinaCommon overseasIntermediate#hybrid-search#rank-fusion

  How to reason about it · think before answering

  1. The hinge word is `scores`. Answering `RRF is simpler` is reciting a concept; the interviewer wants to hear that you know why the two scores are not comparable in the first place.
  2. Start with scale: BM25 is an unbounded sum of log terms, and on one index the top hit can range from 5 to 50 depending on the query; cosine is pinned between -1 and 1. Adding those two readings is meaningless.
  3. Then name the silent failure of normalization: dividing by the per-route maximum makes the denominator float with the query. For a question with no answer in the corpus, the vector route's best hit may score 0.09 and still normalize to a perfect 1.0, entering the fusion at full weight. You think you are comparing relevance; you are comparing `tallest among the short`.
  4. Then the maintenance cost of weights: a 1-to-0.6 ratio has to be tuned against an eval set, tuning two routes is a 2-D search, adding multi-query retrieval makes it 4-D or 5-D, and swapping the embedding model invalidates all of it. RRF has a single k, and the default of 60 rarely needs touching.
  5. Conclusion: rank is the only thing the two routes share. RRF throws the scores away on purpose so that an incomparable quantity cannot mislead it.
  6. Expected follow-up: what does k do? It flattens — the larger k is, the smaller the gap between the top few ranks, so `ranked well by both routes` outweighs `ranked first by one route`, which is exactly the cross-validation effect hybrid retrieval is after. A second follow-up on ties: you must fall back to sorting by document id, or ranks drift between runs and every eval number wobbles with them.

  分析过程 · 先想清楚再作答

  1. 这题的题眼在「分数」两个字。只答「RRF 更简单」是背概念，面试官想听的是你知道分数为什么不可比。
  2. 先给量纲差异：BM25 是一堆对数项累加，没有上界，同一套索引里不同查询的第一名可以从 5 分到 50 分；余弦被钉死在负一到正一。两个读数相加没有意义。
  3. 再点出归一化的静默失败：除以本路最高分之后，分母随查询浮动。一个语料里根本没有答案的问题，向量那一路最高分只有 0.09，归一化之后照样是满分 1.0 带权重进融合——你以为在比相关性，其实在比「本路矮子里有多高」。
  4. 然后是权重的维护成本：1 比 0.6 这个配比要靠跑评估调出来，两路是二维搜索，加上多路查询就是四维五维，而且换一个 embedding 模型全部作废。RRF 只有一个 k，而且 60 这个默认值几乎不用动。
  5. 结论：名次是两路唯一可比的东西。RRF 主动扔掉分数，是为了不被不可比的量误导。
  6. 可预期的追问：那 k 是干什么的？答 k 是压平器——k 越大，头几名之间的差距越小，于是「两路都排进前列」比「一路排第一」更有分量，这正是混合检索想要的交叉验证效果。再追问同分怎么办，答必须按文档 id 兜底排序，否则跨次运行名次会飘、评估数字跟着抖。

  Key points

  • BM25 is unbounded, cosine is bounded; the two scales are not comparable, so adding them is meaningless.
  • Per-route max normalization has a denominator that floats with the query, so the least relevant hit of an unanswerable query still normalizes to 1.0.
  • Weights must be tuned against an eval set, the search is high-dimensional once you add routes, and swapping models invalidates it; RRF has a single constant k.
  • RRF consumes only the ordered id list from each route, because rank is the one thing the routes share.
  • Larger k rewards `ranked well by both routes`; ties must fall back to document id so results are reproducible.

  答题要点

  • BM25 无上界、余弦有界，两个量纲不可比，直接相加没有意义。
  • 按本路最高分归一化的分母随查询浮动，无答案的查询里最不相干的结果也能拿到满分。
  • 权重要跑评估调，路数一多就是高维搜索，换模型还得重来；RRF 只有一个常数 k。
  • RRF 只吃每一路的有序 id 列表，名次是两路唯一可比的东西。
  • k 越大越奖励「两路都排进前列」；同分必须按 id 兜底排序才可复现。
• Why is a cross-encoder more accurate than a bi-encoder? And if it is more accurate, why not just use it to search the whole corpus directly?交叉编码器为什么比双编码器准？既然更准，为什么不干脆拿它直接检索全库？

  Common in ChinaCommon overseasBasic#cross-encoder#bi-encoder

  How to reason about it · think before answering

  1. This is a giveaway question, but the discriminating half is the second part. Saying `cross-encoders are slow` is not enough; you have to point at the structural reason.
  2. Start with the structure: a bi-encoder encodes query and document **separately** into vectors that never meet until a single dot product at the end; a cross-encoder concatenates query and document into one sequence, so every attention layer lets query tokens attend to document tokens.
  3. That yields the accuracy gap: a bi-encoder must compress a document into one fixed-length vector, and compression loses information — the binding between `Zhou Min` and `platform team lead` may not survive. A cross-encoder does not compress; it aligns them on the spot.
  4. The answer to the second half hides in the same structure: bi-encoder document vectors can be computed **offline** and indexed, so query time is just a vector search. A cross-encoder has nothing to precompute — N documents means N forward passes. Reranking a 100k-chunk corpus means pushing the entire corpus through a model on every question.
  5. So the engineering split is a division of labor: recall pulls a small batch out of the whole corpus (cheap, indexable), reranking fixes the order of that batch (expensive, accurate). The default is to rerank only the top 20 after fusion.
  6. Expected follow-up: is there a middle path? Yes — late interaction, where token-level document representations are precomputed and the interaction happens at query time. Accuracy and cost land between the two, at the price of a much larger index.

  分析过程 · 先想清楚再作答

  1. 这是一道送分题，但送分题的区分度在第二问。只答「交叉编码器慢」是不够的，要说清慢在结构上的哪一处。
  2. 先给结构差异：双编码器把查询和文档**各自**编码成向量，两者从头到尾没有见过面，最后只靠一次内积凑到一起；交叉编码器把查询和文档拼成一段文本一起过模型，每一层注意力都能让查询的词去看文档的词。
  3. 由此推出准确率差异的来源：双编码器要把一篇文档压成一个固定长度的向量，压缩必然丢信息，「周敏是平台组组长」里两个词的绑定关系未必留得下来；交叉编码器不压缩，它当场对齐。
  4. 第二问的答案就藏在同一个结构里：双编码器的文档向量**可以离线算好**，查询时只做向量检索；交叉编码器没有任何东西能预先算好，N 篇文档就要跑 N 次前向。十万块的语料重排一遍，等于每次提问都把整个库过一遍模型。
  5. 所以工程上的定位是分工：召回负责在全库里捞出一小批（便宜、可索引），重排负责把这一小批的顺序改对（贵、准）。默认只重排融合后的前 20 条。
  6. 可预期的追问：有没有中间路线？答有——后期交互（late interaction）那一类，文档侧提前算好词级表示、查询侧当场做交互，精度和成本都在两者之间，代价是索引体积大得多。

  Key points

  • A bi-encoder encodes both sides separately and joins them with one dot product; a cross-encoder concatenates them so attention can align across the pair.
  • The accuracy gap comes from compression: a bi-encoder squeezes a whole document into one vector and loses bindings; a cross-encoder does not compress.
  • Bi-encoder document vectors can be computed offline and indexed; a cross-encoder has nothing to precompute.
  • Reranking the full corpus means running every chunk through a model on every question, so cost scales linearly with corpus size.
  • The standard split is recall plus rerank, with reranking applied only to the top few dozen after fusion.

  答题要点

  • 双编码器各自编码、最后一次内积；交叉编码器把查询和文档拼在一起过模型，注意力可以跨两者对齐。
  • 准确率差异来自压缩：双编码器把整篇文档压成一个向量，绑定关系会丢；交叉编码器不压缩。
  • 双编码器的文档向量能离线算好并建索引，交叉编码器没有任何东西可以预先算好。
  • 全库重排等于每次提问把整个语料过一遍模型，成本随语料规模线性增长。
  • 标准分工是召回加重排，重排只作用于融合后的前几十条。
• You replaced pure vector retrieval with hybrid search plus reranking, and after shipping it your eval metrics went down. How do you investigate?你把纯向量检索换成了混合检索加重排，上线之后评估指标反而掉了。你会怎么排查？

  Common in ChinaCommon overseasDeep dive#hybrid-search#evaluation

  How to reason about it · think before answering

  1. This question tests whether you have actually done stage-by-stage attribution. Answering `I would tune the weights and see` loses — that is guessing, not investigating.
  2. Step one is to run the stages apart, not to change code: pure keyword, pure vector, hybrid, and hybrid plus rerank, all on the **same eval set with the same context budget**. Whichever stage the drop appears in is where you look, and this alone separates `fusion is broken` from `reranking is broken`.
  3. Step two asks a specific question: did recall drop, or did the ranking metrics drop? A recall drop means the answer never entered the context at all — a candidate-pool or budget problem. Ranking metrics dropping while recall holds means the answer is still there but pushed down — a fusion-weight or rerank-model problem. The two failures have completely different fixes.
  4. A third common root cause is recall depth. This knob runs against intuition: going deeper is not safer, it lets noise vote too. On a 134-chunk corpus I measured that narrowing each route from 50 to 5 took hybrid recall from 87.5% back to 93.8% and multi-hop from 50% to 75%, while nDCG fell by almost 0.1. The metrics fight each other, so decide which one the product needs first.
  5. A fourth root cause is that the eval protocol quietly changed. Touch the context budget, the hit rule, or the candidate depth, and the old and new numbers stop being comparable — in which case the `drop` may not be a drop at all.
  6. Expected follow-up: how do you avoid this next time? Make the four-way comparison a single command, store the previous report as a baseline, and fail the build with a non-zero exit code on regression. That is precisely why evaluation comes before optimization.

  分析过程 · 先想清楚再作答

  1. 这题考的是你有没有真的做过分阶段归因。答「调一下权重再看看」就输了——那是在猜，不是在查。
  2. 第一步是拆档跑，不是改代码：纯关键词、纯向量、混合、混合加重排四档在**同一份评估集、同一个上下文预算**下各跑一遍。指标掉在哪一档就在哪一档找原因，这一步能立刻区分「融合坏了」和「重排坏了」。
  3. 第二步问一个具体问题：掉的是召回率还是排序指标？召回率掉说明答案根本没进上下文，是候选池或者预算的问题；排序指标掉而召回率没动，说明答案还在、只是被挤到了后面，那是融合权重或重排模型的问题。这两类故障的解法完全不同。
  4. 第三个常见根因是召回深度。每路取多少条这个旋钮方向反直觉：取深了不是更保险，是把噪声也一起投了票。我在一份 134 块的语料上实测过，每路从取 50 收到取 5，混合那一档的召回率从 87.5% 回到 93.8%、多跳档从 50% 回到 75%，而 nDCG 反而掉了近 0.1——两个指标会打架，先想清楚业务要哪个。
  5. 第四个根因是评估口径被悄悄改了。上下文预算、命中判定、候选池深度只要动过一个，新旧数字就不可比，这时候「掉了」可能根本不是真的掉了。
  6. 可预期的追问：怎么防止下次再踩？答把四档对照做成一条命令、把上一版报告存成基线、指标退步就以非 0 退出码拦住合并——这就是评估要先于优化的原因。

  Key points

  • Run all four configurations separately for attribution, on one eval set with one context budget, before touching any parameter.
  • Separate a recall drop from a ranking drop: the first is a candidate-pool or budget issue, the second is a fusion or rerank issue.
  • Check recall depth: taking too many per route lets noise vote, and narrowing it can bring recall back.
  • Confirm the eval protocol did not change; touching budget, hit rule, or candidate depth makes old and new numbers incomparable.
  • Freeze the four-way comparison into one command plus a baseline report, and block merges on regression.

  答题要点

  • 先拆档跑四种配置，在同一份评估集和同一个上下文预算下归因，不要一上来就调参。
  • 区分召回率掉与排序指标掉：前者是候选池或预算问题，后者是融合或重排问题。
  • 查召回深度：每路取太深会把噪声也投进融合，收窄反而可能救回召回率。
  • 确认评估口径没被改：预算、命中判定、候选池深度动过一个，新旧数字就不可比。
  • 把四档对照固化成一条命令加一份基线报告，指标退步直接拦住合并。
• Adding a reranker costs you 200 ms of extra latency per question plus a per-search fee. How do you decide whether that spend is worth it?加上重排之后每次提问多了两百毫秒延迟，还多了一笔按次计费的开销。你怎么判断这笔钱该不该付？

  Common in ChinaCommon overseasDeep dive#rerank#cost-tradeoff

  How to reason about it · think before answering

  1. This question tests whether you can translate a technical choice into a business judgment. Answering `check whether the metrics went up` covers only a third of it.
  2. Split it into three ledgers: how much the metrics moved, how much latency grew, and how much money it costs. All three must be reported together; a proposal with only the first will not survive review.
  3. For the first ledger, be specific about **which** metric reranking improves. Reranking changes the order, not the candidate set — it cannot fix `the answer was never retrieved`. If your recall is the bottleneck, add a retrieval route or adjust recall depth first; the 200 ms buys nothing.
  4. For the second, ask where those 200 ms land. They sit synchronously between retrieval and generation, with the user waiting; but if a streaming generation follows and time-to-first-token is already a second or two, the relative cost is small. In an as-you-type search box, 200 ms is fatal.
  5. For the third, note the billing unit: rerankers usually charge per search rather than per token, so sending a few more candidates barely changes the bill — what is expensive is the number of questions. That points optimization at reducing query volume (caching, intent routing) rather than at trimming the candidate list.
  6. Expected follow-up: what if you simply cannot afford it? Three paths — rerank only queries classified as hard (intent routing), cache results, or self-host an open-weights cross-encoder to convert per-call fees into fixed compute cost.

  分析过程 · 先想清楚再作答

  1. 这题考的是你会不会把技术选择翻译成业务判断。只答「看指标涨没涨」只答了三分之一。
  2. 先把账拆成三笔：指标涨了多少、延迟涨了多少、钱涨了多少。三笔必须一起报，只报第一笔的方案在评审会上过不去。
  3. 第一笔要问清楚重排改善的是**哪个**指标。重排改的是顺序，不是候选集合——它救不了「答案压根没被召回」这种故障。如果你的召回率本来就不够，先去加召回路数或者调召回深度，重排这两百毫秒是白花的。
  4. 第二笔要看这两百毫秒落在哪。它是同步卡在检索之后、生成之前的，用户全程在等；但如果后面接的是一个流式生成、首字节本来就要一两秒，这两百毫秒的相对占比就小得多。反过来，如果这是一个自动补全式的即时搜索框，两百毫秒就是致命的。
  5. 第三笔要注意计价单位：重排普遍按检索次数计价而不是按 token，所以「多送几条给它排」几乎不涨钱，真正贵的是提问次数本身。这直接决定了优化方向是压提问量（缓存、意图路由）而不是压候选数。
  6. 可预期的追问：如果就是付不起怎么办？答三条路——只对判定为复杂的查询走重排（意图路由）、把结果缓存起来、或者换成自部署的开源交叉编码器把按次付费变成固定的算力成本。

  Key points

  • Report all three ledgers together: metric gain, latency growth, cost growth; a proposal missing one is incomplete.
  • Confirm whether the bottleneck is ordering or recall first; reranking only reorders and cannot rescue an answer that was never retrieved.
  • Judge the latency by where it lands: it is small relative to a streaming generation, but fatal in an as-you-type search box.
  • Rerankers bill per search rather than per token, so cost scales with question volume, not candidate count.
  • If it is unaffordable: route only hard queries to the reranker, cache results, or self-host an open-weights cross-encoder.

  答题要点

  • 三笔账一起报：指标增量、延迟增量、成本增量，缺一笔方案就不完整。
  • 先确认瓶颈是排序还是召回：重排只改顺序，救不了没被召回的答案。
  • 延迟要看落在哪：流式生成场景下相对占比小，即时搜索框里两百毫秒就是致命的。
  • 重排按检索次数计价而不是按 token，涨钱的是提问量而不是候选条数。
  • 付不起时的三条路：意图路由只对难查询重排、结果缓存、换自部署的开源交叉编码器。

Comments

Sign in to join the discussion

No comments yet — be the first.