面试题库

14 天 RAG：从检索到可信回答

D1 为什么要检索：幻觉、知识截止与长上下文的代价，以及一个纯关键词的最小 RAG

• 上下文窗口已经做到上百万 token 了，检索这一步会被淘汰吗？Context windows are now in the millions of tokens. Does that make the retrieval step obsolete?

  国内高频海外高频深入#long-context#cost#system-design

  分析过程 · 先想清楚再作答

  1. 这是一道立场题，容易答成非黑即白。判断你有没有做过的地方在于：会不会区分「技术上能不能塞进去」和「工程上该不该每次都塞」，只谈前者的答案一听就是纸上谈兵。
  2. 先承认对方有道理的部分：窗口变大确实吃掉了检索的一部分场景。几十篇文档、更新不频繁、调用量不大的内部工具，直接全塞是最省事的选择，为它建一套检索系统是过度设计。
  3. 再给三条它吃不掉的理由。第一是成本：材料是按次计费的，同一份材料被问一万次就要付一万次，而检索只付取回的那几段；预填充缓存能缓解但不能消除，缓存也有有效期和命中率。
  4. 第二是规模：企业知识库动辄几十万篇，再大的窗口也塞不下，检索是唯一的入口。第三是归因与权限：答案要指回具体某一段，以及不同的人只能看到自己有权访问的材料——这两件事必须在把材料喂给模型之前完成，窗口再大也不解决。
  5. 还要补一条经验事实：材料变多之后，模型在长上下文里定位关键信息的稳定性会下降，出现「读了但没读到」。所以「全塞」并不总是等于「效果更好」，很多时候少而准反而更好。
  6. 可预期的追问：那检索的形态会不会变？会——窗口变大之后，取回的块可以更大、条数可以更多，重排与压缩的压力变小，检索从「精挑几句」变成「粗筛一批」。趋势是检索的粒度变粗，不是检索消失。

  How to reason about it · think before answering

  1. This is a position question and it is easy to answer as a binary. The signal is whether you separate what fits technically from what is worth paying for on every request.
  2. Concede the valid half first: bigger windows genuinely absorb part of the use case. For an internal tool over a few dozen stable documents with low traffic, stuffing everything in is the right call and building a retrieval stack would be over-engineering.
  3. Then give three reasons it does not absorb the rest. Cost is the first: context is billed per request, so the same corpus is paid for on every one of ten thousand queries, whereas retrieval only pays for the passages it returns. Prompt caching softens this but does not remove it.
  4. Scale is the second: enterprise corpora run to hundreds of thousands of documents and no window holds them. Attribution and access control are the third: pointing an answer at a specific passage, and showing each user only what they are permitted to see, both have to happen before the material reaches the model.
  5. Add the empirical point: as the supplied material grows, models become less reliable at locating the one relevant fact inside it. More context is not automatically better; fewer and more precise passages often win.
  6. Expected follow-up: does retrieval change shape? Yes. Larger windows allow bigger chunks and more of them, which relieves pressure on reranking and compression. Retrieval gets coarser, it does not disappear.

  答题要点

  • 先区分「能不能塞进去」和「该不该每次都塞」，前者是技术问题，后者是成本问题。
  • 小规模、低频、少变的语料确实可以直接全塞，为它建检索系统是过度设计。
  • 检索不会被淘汰的三个理由：按次计费的成本、几十万篇塞不下的规模、必须在喂给模型之前完成的归因与权限过滤。
  • 材料越多，模型定位关键信息的稳定性越差，全塞不等于效果更好。
  • 趋势是检索粒度变粗——块更大、条数更多、重排压力变小，而不是检索消失。

  Key points

  • Separate whether it fits from whether it is worth paying for on every request.
  • Small, stable, low-traffic corpora can legitimately be stuffed whole; building retrieval for them is over-engineering.
  • Three reasons retrieval survives: per-request cost, corpora too large for any window, and attribution plus access control that must happen before the model sees the material.
  • More supplied context reduces the reliability of locating a single fact, so stuffing everything is not automatically better.
  • The trend is coarser retrieval — bigger chunks, more of them, less reranking pressure — not the removal of retrieval.

D14 综合项目与复盘：多租户企业知识库问答，一张 RAG 决策地图与面试专题

• 给你一个五百万文档、要求秒级响应、准确率优先的知识库场景，你会怎么设计这套系统？You are handed a knowledge base of five million documents that must answer in about a second, with accuracy as the top priority. How would you design it?

  国内高频海外高频深入#system-design#scaling#latency-budget

  分析过程 · 先想清楚再作答

  1. 这题的题眼不在「你会用什么技术」，而在「你有没有一套从约束推配置的方法」。开口就报架构图和技术栈的答案会被判成背方案；拿到分的答法是先把约束翻译成数字，再让每个选择被某个数字逼出来。
  2. 先把三个约束量化：五百万文档按一篇四五块估，是两千多万块，单精度 1536 维就是上百 GB，**索引塞不进单机内存**，这一条直接决定了存储选型；秒级响应意味着从收到问题到第一个字的预算大约一秒，而生成本身通常就吃掉七八百毫秒，检索侧只剩两三百毫秒；准确率优先意味着可以拿延迟和钱换指标，但只能换到那两三百毫秒为止。
  3. 然后逐项落地，每一项都挂在上面某个数字上：存储上专用向量库或分区加半精度量化（半精度的召回损失通常落在重跑噪声里，索引却小四成，这是白捡的）；检索保留关键词与向量两路加倒数排名融合，因为精确匹配的文档号、错误码、人名是向量的固定盲区；重排只作用于融合后的前二十条——它买的是排序质量，一次同步往返，秒级预算里放得下一次，放不下两次。
  4. 接着讲两个「不上」的决定，这一段比上面更能显出做过工程：**Agentic 检索不作为默认路径**，它的收益集中在多跳题上而代价摊给全部问题，秒级预算下更是直接超支——正确做法是先用一次便宜的分类把多跳分流出来，只让那一小部分进循环；**上下文块头之类的手法先不上**，因为它对关键词一路是稀释、对向量一路才是补位，方向相反，得在自己的真实 embedding 上测过再说。
  5. 准确率优先必须落成可验收的东西，否则是空话：一份不少于一百题的标准答案集（其中多跳与无答案各占一成以上）、召回率与排序质量分开看、无答案题的拒答率单独一栏、引用由代码回查而不是靠提示词自觉。**报数字时把最难看的那一栏也报出来**，比只报总分可信得多。
  6. 可预期的追问：五百万文档怎么建第一版索引？答案是这笔钱是一次性大额支出，要按批做、可断点续跑，并且从第一天就上基于内容指纹的增量同步——否则每次改配置都等于把整个知识库重买一遍。再追问就谈灰度：新旧两套向量双写在两列上，用同一份标准答案集在两列上各跑一遍再切流量，回滚只是改一个配置项。

  How to reason about it · think before answering

  1. The real subject here is not which technologies you know, it is whether you have a repeatable way to derive a configuration from constraints. Opening with an architecture diagram reads as a memorized answer; the way to score is to turn each constraint into a number first, then let every choice be forced by one of those numbers.
  2. Quantify the three constraints. Five million documents at roughly four or five chunks each is over twenty million chunks; at 1536 float dimensions that is hundreds of gigabytes, so the index does not fit in one machine's memory — that alone settles storage. A one-second budget to first token, with generation typically eating seven or eight hundred milliseconds, leaves only two or three hundred for retrieval. Accuracy first means you may trade latency and money for metrics, but only within that remaining budget.
  3. Now derive each knob from one of those numbers: a dedicated vector store or partitioning, plus half precision (its recall loss usually sits inside run-to-run noise while the index shrinks by about forty percent — essentially free); keep both keyword and vector routes with reciprocal rank fusion, because exact matches on document ids, error codes and names are a permanent blind spot for embeddings; rerank only the top twenty after fusion, since it buys ranking quality at the cost of one synchronous round trip, and a one-second budget affords exactly one.
  4. Then state two things you deliberately do not build, which is the part that reads as field experience. Agentic retrieval is not the default path: its gains concentrate on multi-hop questions while its cost is spread over every question, and it blows a one-second budget outright — the right move is a cheap classifier that routes only the multi-hop minority into the loop. Contextual chunk headers and similar tricks also wait, because they dilute the keyword route while helping the vector route; the directions are opposite, so measure on your own embeddings before committing.
  5. Accuracy first has to become something you can sign off on. That means a golden set of at least a hundred questions with multi-hop and unanswerable each above ten percent, recall and ranking quality read separately, abstention rate on unanswerable questions as its own column, and citations verified by code rather than trusted from the model. Reporting the ugliest column alongside the headline number is far more credible than reporting a single score.
  6. Expected follow-up: how do you build the first index over five million documents? It is a one-off large expense, so batch it, make it resumable, and put content-hash incremental sync in from day one, or every config change means buying the whole corpus again. Push further and you get to rollout: dual-write the new embeddings into a second column, evaluate both columns on the same golden set, then shift traffic, so rollback is a config flip rather than an eight-hour rebuild.

  答题要点

  • 先把约束翻译成数字：两千多万块决定索引塞不进单机内存，一秒预算里检索侧只剩两三百毫秒。
  • 存储用专用库或分区加半精度；检索保留关键词与向量两路加倒数排名融合，重排只作用于前二十条。
  • 明确说出「不上」的两项：Agentic 只对分流出来的多跳开，块头这类方向相反的手法先测再说。
  • 准确率优先要落成一百题以上的标准答案集、拒答率单独一栏、引用由代码回查。
  • 第一版建索引是一次性大额支出：分批可续跑，并从第一天就上增量同步；换模型走双写切列。

  Key points

  • Translate constraints into numbers first: twenty million chunks means the index will not fit one machine, and a one-second budget leaves retrieval two to three hundred milliseconds.
  • Dedicated store or partitions plus half precision; keep keyword and vector routes with RRF, and rerank only the top twenty after fusion.
  • Name the two things you will not ship: agentic only for a routed multi-hop minority, and chunk headers only after measuring on your own embeddings.
  • Turn accuracy-first into a hundred-plus question golden set, abstention rate as its own column, and code-verified citations.
  • First index build is a one-off large expense: batch it, make it resumable, add incremental sync on day one, and dual-write columns for model swaps.