面试题库

14 天 RAG：从检索到可信回答

D2 embedding 与向量检索：相似度、维度与模型选型，把文本存进 pgvector

• 向量检索能完全取代关键词检索吗？举一个向量必然失手的查询，并说说你会怎么补。Can vector search fully replace keyword search? Give a query where vectors are bound to fail, and say how you would fix it.

  国内高频海外高频进阶#hybrid-search#embeddings#retrieval-failure

  分析过程 · 先想清楚再作答

  1. 这题是典型的「立场题」，答「能」或「不能」都不重要，重要的是你能不能举出一个具体到能复现的反例。举不出例子，前面说得再漂亮也会被判成没做过。
  2. 先给失手的类型，一次给全：错误码与状态码（429、E1032）、版本号与型号（v2.3.1、X20 Pro）、人名与工号、订单号与文档编号、以及否定表达。前四类的共同点是**这些词的价值在于字面唯一，而向量只保留语义邻近**，模型会把 429 和「限流」「超时」这些话题相近的东西编到一起，反而把真正写着 429 的那篇挤下去。
  3. 拿一个能复现的例子说：问「限流超了返回 429 吗」，BM25 稳稳命中写着 429 的接口文档，向量却可能把话题相近但没提 429 的产品手册排在前面。这个现象在本课第 2 天的实验里就能亲眼看到。
  4. 否定表达要单独强调：「支持导出 PDF」和「不支持导出 PDF」在向量空间里几乎重合，因为它们谈的是同一件事。指望向量区分肯定与否定一定翻车，这一层要靠生成侧读原文来判断。
  5. 怎么补：两路并行跑再融合，关键词一路用 BM25、向量一路用最近邻，用倒数排名融合把两个名次合成一个。这就是混合检索，本课第 9 天展开。要点是**两套的错法不一样**，所以合起来才有增益——如果两套错在同一批查询上，融合是白做的。
  6. 可预期的追问：那关键词一路能不能扔掉、改成让模型改写查询？可以缓解一部分（第 10 天的查询改写），但改写救不了字面唯一的标识符——你没法把 429 改写成别的说法。

  How to reason about it · think before answering

  1. This is a stance question where the stance matters less than the counter-example. Without a concrete, reproducible failing query, the rest of the answer reads as theory.
  2. Enumerate the failure classes up front: error and status codes, version numbers and SKUs, names and employee IDs, order or document identifiers, and negation. The first four share one property: their value lies in exact literal identity, which embeddings deliberately blur into semantic neighbourhoods.
  3. Give a reproducible example: ask whether rate limiting returns 429. BM25 lands on the API document that literally contains 429, while vector search may rank a topically similar product manual that never mentions the code.
  4. Call out negation separately: 'supports PDF export' and 'does not support PDF export' sit almost on top of each other because they discuss the same thing. Vectors cannot carry that distinction; the generation step reading the source has to.
  5. The fix: run both retrievers and fuse the rankings, BM25 on the lexical side and nearest neighbour on the vector side, combined with reciprocal rank fusion. That is hybrid search, covered on day 9. Fusion helps precisely because the two systems fail on different queries.
  6. Expected follow-up: could you drop the keyword path and rewrite queries instead? Rewriting helps with vocabulary mismatch, but it cannot rescue exact identifiers, since there is no paraphrase of 429.

  答题要点

  • 不能取代：错误码、版本号、人名、单号这类词的价值在于字面唯一，向量只保留语义邻近。
  • 具体反例：问「限流超了返回 429 吗」，BM25 命中写着 429 的文档，向量把话题相近却没提 429 的文档排前面。
  • 否定表达是另一类失手：肯定句与否定句在向量空间里几乎重合。
  • 补法是混合检索：两路并行再用倒数排名融合合并名次。
  • 融合有增益的前提是两套的错法不同；查询改写能缓解词汇不匹配，但救不了字面唯一的标识符。

  Key points

  • No: codes, version numbers, names and IDs matter as exact literals, which embeddings blur into neighbourhoods.
  • Concrete example: asking whether rate limiting returns 429, where BM25 hits the document containing 429 and vectors surface a topically similar one that never mentions it.
  • Negation is a second failure class, since affirmative and negative statements sit almost on top of each other.
  • The remedy is hybrid retrieval: run both paths and merge with reciprocal rank fusion.
  • Fusion pays off because the two paths fail differently; query rewriting helps vocabulary mismatch but not exact identifiers.

D9 混合检索与重排：两路召回、倒数排名融合，再用交叉编码器把前几名重新排一遍

• 混合检索为什么普遍用倒数排名融合，而不是把两路分数归一化之后加权相加？加权那条路在什么情况下会失控？Why do hybrid retrieval systems usually use reciprocal rank fusion instead of normalizing both scores and adding them with weights? When does the weighted approach break down?

  国内高频海外高频进阶#hybrid-search#rank-fusion

  分析过程 · 先想清楚再作答

  1. 这题的题眼在「分数」两个字。只答「RRF 更简单」是背概念，面试官想听的是你知道分数为什么不可比。
  2. 先给量纲差异：BM25 是一堆对数项累加，没有上界，同一套索引里不同查询的第一名可以从 5 分到 50 分；余弦被钉死在负一到正一。两个读数相加没有意义。
  3. 再点出归一化的静默失败：除以本路最高分之后，分母随查询浮动。一个语料里根本没有答案的问题，向量那一路最高分只有 0.09，归一化之后照样是满分 1.0 带权重进融合——你以为在比相关性，其实在比「本路矮子里有多高」。
  4. 然后是权重的维护成本：1 比 0.6 这个配比要靠跑评估调出来，两路是二维搜索，加上多路查询就是四维五维，而且换一个 embedding 模型全部作废。RRF 只有一个 k，而且 60 这个默认值几乎不用动。
  5. 结论：名次是两路唯一可比的东西。RRF 主动扔掉分数，是为了不被不可比的量误导。
  6. 可预期的追问：那 k 是干什么的？答 k 是压平器——k 越大，头几名之间的差距越小，于是「两路都排进前列」比「一路排第一」更有分量，这正是混合检索想要的交叉验证效果。再追问同分怎么办，答必须按文档 id 兜底排序，否则跨次运行名次会飘、评估数字跟着抖。

  How to reason about it · think before answering

  1. The hinge word is `scores`. Answering `RRF is simpler` is reciting a concept; the interviewer wants to hear that you know why the two scores are not comparable in the first place.
  2. Start with scale: BM25 is an unbounded sum of log terms, and on one index the top hit can range from 5 to 50 depending on the query; cosine is pinned between -1 and 1. Adding those two readings is meaningless.
  3. Then name the silent failure of normalization: dividing by the per-route maximum makes the denominator float with the query. For a question with no answer in the corpus, the vector route's best hit may score 0.09 and still normalize to a perfect 1.0, entering the fusion at full weight. You think you are comparing relevance; you are comparing `tallest among the short`.
  4. Then the maintenance cost of weights: a 1-to-0.6 ratio has to be tuned against an eval set, tuning two routes is a 2-D search, adding multi-query retrieval makes it 4-D or 5-D, and swapping the embedding model invalidates all of it. RRF has a single k, and the default of 60 rarely needs touching.
  5. Conclusion: rank is the only thing the two routes share. RRF throws the scores away on purpose so that an incomparable quantity cannot mislead it.
  6. Expected follow-up: what does k do? It flattens — the larger k is, the smaller the gap between the top few ranks, so `ranked well by both routes` outweighs `ranked first by one route`, which is exactly the cross-validation effect hybrid retrieval is after. A second follow-up on ties: you must fall back to sorting by document id, or ranks drift between runs and every eval number wobbles with them.

  答题要点

  • BM25 无上界、余弦有界，两个量纲不可比，直接相加没有意义。
  • 按本路最高分归一化的分母随查询浮动，无答案的查询里最不相干的结果也能拿到满分。
  • 权重要跑评估调，路数一多就是高维搜索，换模型还得重来；RRF 只有一个常数 k。
  • RRF 只吃每一路的有序 id 列表，名次是两路唯一可比的东西。
  • k 越大越奖励「两路都排进前列」；同分必须按 id 兜底排序才可复现。

  Key points

  • BM25 is unbounded, cosine is bounded; the two scales are not comparable, so adding them is meaningless.
  • Per-route max normalization has a denominator that floats with the query, so the least relevant hit of an unanswerable query still normalizes to 1.0.
  • Weights must be tuned against an eval set, the search is high-dimensional once you add routes, and swapping models invalidates it; RRF has a single constant k.
  • RRF consumes only the ordered id list from each route, because rank is the one thing the routes share.
  • Larger k rewards `ranked well by both routes`; ties must fall back to document id so results are reproducible.