Interview Bank

How to reason about it · think before answering

1. The lazy answer is 'sort again, more accurately'. What the interviewer wants is why the first pass cannot rank well, and why reranking cannot run over the whole corpus.
2. Decompose: the first pass ranks by retrieval signals — cosine distance or term statistics — which are designed to scan millions of items fast, and coarseness is the price. Reranking changes the algorithm: query and candidate go into one model together (a cross-encoder), which is far more accurate but costs one forward pass per candidate. Hence it must sit behind a wide recall stage.
3. Distinguish two implementations. For teaching or prototypes, batch-score with an LLM (0-10 for 40 candidates in one call). Production uses a trained cross-encoder reranker. Name the cost: an extra 100-300 ms hop plus an inference box — it is not a per-token API, it consumes capacity.
4. Framing it as a funnel is clearest: recall sets the ceiling, reranking decides whether what is under the ceiling reaches the top five. Measured: adding the keyword channel lifts recall@20 from 83% to 95%; adding reranking moves recall@20 only to 98%, but recall@5 jumps from 80% to 91% and MRR from 0.732 to 0.908.
5. Expected follow-up 1: does reranking improve recall? No. It introduces no new candidates, so recall@20 is the wrong metric to judge it by.
6. Expected follow-up 2: why not ship LLM scoring to production? Unpredictable latency, per-token cost, scores that drift with prompt wording, and no clean path to offline distillation.

分析过程 · 先想清楚再作答

1. 这题最容易答成「再排一次序，更准」。面试官想听的是**为什么第一轮不能直接排准**，以及**为什么重排不能对全库做**。
2. 怎么拆：第一轮的排序依据是「检索信号」——余弦距离或词频统计，它们是为了能在百万条里快速筛选而设计的，代价就是粗。重排换了一种算法：把 query 和候选**拼在一起**送进同一个模型算相关度（cross-encoder），精度高得多，但复杂度是每条候选一次前向，没法对全库做。所以它必须跟在一个宽召回后面。
3. 结论要区分两种实现：教学 / 原型可以用 LLM 批量打分（一次调用给 40 条打 0 到 10 分），生产用专门训练的 cross-encoder 重排模型。**代价说清楚：多一次 100 到 300 毫秒的调用，外加一台推理机器**——它不是按 token 计费的 API，是要占资源的。
4. 把它放进漏斗里说最清楚：召回决定天花板，重排决定天花板上的东西能不能排到前五。实测的样子是——加了关键词那一路，recall@20 从 83% 涨到 95%（天花板抬高）；再加重排，recall@20 只到 98%，但 recall@5 从 80% 跳到 91%、MRR 从 0.732 到 0.908。
5. 可预期的追问一：重排能不能提高召回？不能。它不引入新候选，只重排已有的那批——所以看 recall@20 判断重排效果是错的指标。
6. 可预期的追问二：为什么不用 LLM 打分上生产？延迟不可控、成本按 token 走、分数会随提示词措辞漂移，而且没法做批量离线蒸馏。

How to reason about it · think before answering

1. This question tests whether you can translate a technical choice into a business judgment. Answering `check whether the metrics went up` covers only a third of it.
2. Split it into three ledgers: how much the metrics moved, how much latency grew, and how much money it costs. All three must be reported together; a proposal with only the first will not survive review.
3. For the first ledger, be specific about **which** metric reranking improves. Reranking changes the order, not the candidate set — it cannot fix `the answer was never retrieved`. If your recall is the bottleneck, add a retrieval route or adjust recall depth first; the 200 ms buys nothing.
4. For the second, ask where those 200 ms land. They sit synchronously between retrieval and generation, with the user waiting; but if a streaming generation follows and time-to-first-token is already a second or two, the relative cost is small. In an as-you-type search box, 200 ms is fatal.
5. For the third, note the billing unit: rerankers usually charge per search rather than per token, so sending a few more candidates barely changes the bill — what is expensive is the number of questions. That points optimization at reducing query volume (caching, intent routing) rather than at trimming the candidate list.
6. Expected follow-up: what if you simply cannot afford it? Three paths — rerank only queries classified as hard (intent routing), cache results, or self-host an open-weights cross-encoder to convert per-call fees into fixed compute cost.

分析过程 · 先想清楚再作答

1. 这题考的是你会不会把技术选择翻译成业务判断。只答「看指标涨没涨」只答了三分之一。
2. 先把账拆成三笔：指标涨了多少、延迟涨了多少、钱涨了多少。三笔必须一起报，只报第一笔的方案在评审会上过不去。
3. 第一笔要问清楚重排改善的是**哪个**指标。重排改的是顺序，不是候选集合——它救不了「答案压根没被召回」这种故障。如果你的召回率本来就不够，先去加召回路数或者调召回深度，重排这两百毫秒是白花的。
4. 第二笔要看这两百毫秒落在哪。它是同步卡在检索之后、生成之前的，用户全程在等；但如果后面接的是一个流式生成、首字节本来就要一两秒，这两百毫秒的相对占比就小得多。反过来，如果这是一个自动补全式的即时搜索框，两百毫秒就是致命的。
5. 第三笔要注意计价单位：重排普遍按检索次数计价而不是按 token，所以「多送几条给它排」几乎不涨钱，真正贵的是提问次数本身。这直接决定了优化方向是压提问量（缓存、意图路由）而不是压候选数。
6. 可预期的追问：如果就是付不起怎么办？答三条路——只对判定为复杂的查询走重排（意图路由）、把结果缓存起来、或者换成自部署的开源交叉编码器把按次付费变成固定的算力成本。