面试题库

14 天 RAG：从检索到可信回答

D1 为什么要检索：幻觉、知识截止与长上下文的代价，以及一个纯关键词的最小 RAG

• BM25 里的词频饱和与文档长度归一化分别在解决什么问题？把 k1 和 b 都设成 0 会发生什么？In BM25, what problems do term-frequency saturation and document length normalisation each solve? What happens if you set both k1 and b to zero?

  国内高频海外高频进阶#bm25#ranking#information-retrieval

  分析过程 · 先想清楚再作答

  1. 这题考的是你有没有真的读过公式，而不是有没有调过库。判据很明确：能不能把 k1 和 b 各自对应到公式里的哪一项，并说出去掉之后会被什么样的文档钻空子。
  2. 先说朴素词频的两个漏洞：一是重复刷词，一篇文章把关键词写五十遍就能霸榜；二是长文占便宜，文档越长越容易蒙中查询里的词。这两个漏洞正好对应两个修正。
  3. k1 管第一个漏洞。分子分母里都有词频 f，所以词频涨上去之后整个分式趋近一个上界而不是线性增长——写五十遍确实比写五遍相关，但绝不该相关十倍。k1 越小饱和越快。
  4. b 管第二个漏洞。归一化项是 1 减 b 加上 b 乘以本文长度除以平均长度，b 等于 0 时完全不看长度，b 等于 1 时完全按长度比例惩罚，0.75 是长期折中的默认值。
  5. 回到题干那个陷阱：k1 设成 0 会让分式退化成常数，词出现一次和一百次得分完全一样，等于只剩「有没有出现过」的布尔匹配；b 设成 0 则长度信息彻底消失。两个一起设成 0，BM25 就退化成对逆文档频率求和，跟词频再无关系。
  6. 可预期的追问：那逆文档频率去掉行不行？答案是不行，去掉之后「的」「我们」这类高频词会淹没一切——而且要顺带说明 BM25 因此天然不需要停用词表，这一句最能体现你读懂了公式。

  How to reason about it · think before answering

  1. This checks whether you have actually read the formula rather than merely called a library. The test is whether you can map k1 and b onto specific terms and name the failure each one prevents.
  2. Start with the two holes in raw term frequency: keyword stuffing lets one document dominate by repeating a word, and long documents win by accident because they contain more words overall.
  3. k1 closes the first hole. Term frequency appears in both numerator and denominator, so the ratio approaches a ceiling instead of growing linearly. Fifty mentions are more relevant than five, but not ten times more relevant. A smaller k1 saturates sooner.
  4. b closes the second. The normalisation factor is one minus b plus b times document length over average length: at b equal to zero length is ignored entirely, at one it is fully penalised, and 0.75 is the conventional compromise.
  5. Now the trap in the question: k1 equal to zero collapses the ratio to a constant, so one occurrence scores the same as a hundred and matching becomes boolean. b equal to zero removes length entirely. Set both to zero and BM25 degenerates into a plain sum of inverse document frequencies.
  6. Expected follow-up: can you drop the IDF term? No. Without it, ubiquitous words drown everything else, and it is precisely IDF that lets BM25 work without a stopword list.

  答题要点

  • 词频饱和由 k1 控制，防的是重复刷词：词频涨大后得分趋近上界而非线性增长。
  • 长度归一化由 b 控制，防的是长文档靠词多蒙中查询，用本文长度比平均长度把它压回去。
  • k1 设 0 会退化成布尔匹配，词出现一次和一百次同分；b 设 0 则完全不考虑文档长度。
  • 两者都设 0 时 BM25 只剩逆文档频率求和，等于放弃了词频信息。
  • 逆文档频率是第三块，让稀有词权重更高，也让 BM25 天然不需要停用词表。

  Key points

  • k1 controls saturation and prevents keyword stuffing: the score approaches a ceiling rather than growing linearly with frequency.
  • b controls length normalisation and stops long documents from winning by sheer word count.
  • Setting k1 to zero degenerates the scorer into boolean matching; one occurrence scores the same as a hundred.
  • Setting b to zero removes document length from the equation entirely; both at zero leaves only a sum of IDF terms.
  • IDF is the third component: it up-weights rare terms and removes the need for a stopword list.
• 一个检索增强生成系统答错了，你怎么定位是检索的锅还是生成的锅？A retrieval-augmented generation system gave a wrong answer. How do you determine whether retrieval or generation is at fault?

  国内高频海外高频进阶#debugging#failure-modes#evaluation

  分析过程 · 先想清楚再作答

  1. 题眼在「怎么定位」，不在「有哪些原因」。答成一串可能原因的罗列就输了，面试官想听的是一个有先后顺序、能落到具体动作的排查流程。
  2. 先给最省时间的第一步：把这次检索出来的几段原文原样打印出来，自己读一遍。正确答案不在里面就是检索的锅，在里面而模型没用上才是生成的锅。这一步三十秒，能省掉大半天的瞎猜。
  3. 然后把链路展开成五个环节——切块、建索引、检索、组装上下文、生成——并给出「排查从右往左、修复从左往右」这条口径：从右往左是因为你最先看到的是生成结果，从左往右是因为左边的错会被右边放大。
  4. 补充几个能把环节钉死的症状：答案「半对」多半是切块把一条完整规则切断了；检索结果里混着一眼不相干的东西多半是解析没做干净；模型无视材料用先验知识作答，通常是提示词里少了「只能依据资料回答」；引用编号和内容对不上，那是生成侧漏读或串了行。
  5. 最后落到工程做法：这套排查要能重复做，就必须把每次请求的检索结果、进上下文的段落、最终回答一起记下来，否则线上出问题时你根本复现不了。到了要批量做的时候，就得换成一批固定问题加指标，而不是一条条人工看。
  6. 可预期的追问：如果检索确实没捞到，改提示词有没有用？答案是没用——材料里没有的东西，再好的指令也只能换一种编法。这句话最能证明你分清了两层。

  How to reason about it · think before answering

  1. The question asks how you localise the fault, not what the possible causes are. Listing causes loses; the interviewer wants an ordered procedure that ends in concrete actions.
  2. Give the cheapest first step: print the retrieved passages verbatim and read them. If the correct answer is not in there, retrieval is at fault. If it is in there and the model ignored it, generation is at fault. Thirty seconds, and it removes most of the guesswork.
  3. Then lay out the five stages — chunking, indexing, retrieval, context assembly, generation — with the rule: diagnose right to left, fix left to right. You see the generated answer first, but an error on the left is amplified by everything to its right.
  4. Add symptoms that pin down a stage: half-correct answers usually mean a rule was split across chunks; obviously irrelevant hits usually mean dirty parsing; the model ignoring the supplied material usually means the prompt never said it must; citation numbers that do not match their content point at generation.
  5. Land it in engineering terms: to run this procedure repeatedly you must log the retrieved hits, the passages that entered the context, and the final answer together, otherwise production issues are unreproducible. At scale this becomes a fixed question set with metrics rather than case-by-case reading.
  6. Expected follow-up: if retrieval missed the document, will prompt tuning help? No. Nothing in the prompt can conjure material that was never supplied.

  答题要点

  • 第一步永远是把检索出来的原文打印出来读一遍，判断正确答案在不在里面。
  • 把链路拆成切块、建索引、检索、组装上下文、生成五个环节，排查从右往左、修复从左往右。
  • 用症状钉环节：半对多半是切块问题，混入无关结果多半是解析问题，无视材料多半是提示词缺约束，引用与内容对不上是生成问题。
  • 检索没捞到时改提示词没有意义，材料里没有的东西模型只能编。
  • 要能重复排查就必须把检索结果、进上下文的段落和最终回答一起记录下来。

  Key points

  • Always start by printing the retrieved passages and checking whether the correct answer is present at all.
  • Split the pipeline into chunking, indexing, retrieval, context assembly and generation; diagnose right to left, fix left to right.
  • Use symptoms to pin the stage: half-correct answers point at chunking, irrelevant hits at parsing, ignored material at the prompt, mismatched citations at generation.
  • If retrieval missed the document, prompt changes cannot help; the material simply is not there.
  • Log retrieved hits, the passages that entered the context, and the final answer together, or production failures are unreproducible.

D2 embedding 与向量检索：相似度、维度与模型选型，把文本存进 pgvector

• 把 embedding 维度从 1536 降到 512，你会损失什么？什么场景下这个损失可以接受？What do you lose when you cut embedding dimensions from 1536 to 512, and when is that loss acceptable?

  国内高频海外高频进阶#embeddings#dimensions#cost

  分析过程 · 先想清楚再作答

  1. 这题考的是你会不会算账。只说「维度越低越省、精度越低」的答案没有区分度，面试官在等一个具体的成本模型和一个决策顺序。
  2. 先把三笔账列出来：存储与内存（向量数量乘维度乘每维字节数，近似最近邻索引要把它放进内存，所以基本等于机器预算）、检索延迟（每次比较就是一轮乘加，维度大致线性影响耗时）、检索质量（收益递减，低维段每加一档提升明显，高维段加倍只换来很小的改善）。
  3. 再说清降维为什么可行：主流模型用套娃式表示训练，重要信息压在靠前的维度上，所以直接截短再归一化仍然可用，这不是另训了一个小模型。截短必然有损失，损失多少只能在自己的数据上跑评估才知道。
  4. 给出决策顺序：先按存储与内存预算倒推一个维度上限，再从上限往下试两三档，看指标掉多少，掉得能接受就用低的。反过来「先选最高维再想办法省钱」基本都会返工。
  5. 点出可接受的典型场景：库很大而单条价值不高（比如日志、工单）、召回之后还有重排兜底（重排能把粗排的损失补回来一部分）、或者对延迟极敏感的在线场景。反过来法务、医疗这类一条都不能漏的场景就要谨慎。
  6. 可预期的追问：能不能不同文档用不同维度？不能——同一个索引里所有向量必须同维，改维度等于全库重建，这跟换模型是同一类迁移成本。

  How to reason about it · think before answering

  1. This is a cost-modelling question. 'Lower dimensions are cheaper but less accurate' earns nothing; the interviewer wants a cost model and a decision order.
  2. Lay out three costs: storage and memory (vector count times dimensions times bytes per dimension, which an ANN index must hold in RAM), query latency (roughly linear in dimensions), and retrieval quality, whose returns diminish sharply at the high end.
  3. Explain why truncation works at all: models trained with Matryoshka representations pack the most important information into the leading dimensions, so truncating and re-normalising keeps the vector usable. It is still lossy, and how lossy is an empirical question on your own data.
  4. Give the decision order: derive a dimension ceiling from your memory budget, then step down two or three notches and measure the metric drop. Choosing the largest model first and optimising cost later usually means redoing the work.
  5. Name the acceptable cases: large corpora of low individual value, pipelines where a reranker recovers some of the loss, and latency-critical online paths. Be conservative where a single miss is expensive, such as legal or clinical retrieval.
  6. Expected follow-up: can different documents use different dimensions? No. Every vector in an index must share one dimension, so changing it means rebuilding the whole index, the same migration cost as changing models.

  答题要点

  • 三笔账：存储与索引内存、检索延迟、检索质量，前两笔随维度近似线性，第三笔收益递减。
  • 套娃式表示让截短再归一化仍然可用，但一定有损失，损失多少要在自己的数据上评估。
  • 决策顺序是先按内存预算定上限，再往下试档位看指标掉多少。
  • 库大、单条价值低、后面还有重排兜底、对延迟敏感的场景，降维划算。
  • 同一索引里维度必须一致，改维度等于全库重建。

  Key points

  • Three costs: storage and index memory, query latency, and retrieval quality; the first two scale with dimensions, the third has diminishing returns.
  • Matryoshka representations make truncation viable, but it is lossy and the loss must be measured on your own data.
  • Decide by deriving a ceiling from the memory budget, then stepping down and measuring.
  • Truncation pays off for large corpora, low-value items, latency-sensitive paths, and pipelines with a reranker.
  • All vectors in one index share a dimension, so changing it forces a full rebuild.
• 为什么有些 embedding 模型要求查询和文档加不同的前缀？不加会怎样，你怎么在上线前发现这个问题？Why do some embedding models require different prefixes for queries and documents? What happens if you skip them, and how would you catch it before shipping?

  国内高频海外高频进阶#embeddings#model-selection#evaluation

  分析过程 · 先想清楚再作答

  1. 这题的题眼是「静默失效」。会背「e5 要加 query 和 passage 前缀」只能拿基础分，能说清它为什么不报错、以及怎么在上线前抓住它，才是做过的人。
  2. 先讲原因：这一族模型是拿成对数据训练的，一侧是短问句、一侧是长段落，两者的分布本来就不一样。前缀是训练时给模型的角色标记，告诉它这一段该按查询编码还是按文档编码。推理时不给，模型就落在了训练分布之外。
  3. 再讲后果的性质：不加前缀模型照样输出向量、照样能算距离、名次照样有先后，只是整体质量下滑。**没有任何报错**——这跟忘了归一化是同一类问题：错误不会自己浮出来。
  4. 怎么发现：唯一可靠的办法是一小份标注问题集，用同一批文档跑两遍（加前缀与不加前缀），比命中率。这就是第 8 天要做的评估闸门，它的价值恰恰在于抓这类静默错误。上线前跑一遍，比读十遍文档管用。
  5. 补一个更容易踩的变体：**建库时加了前缀、查询时忘了加**，或者两边加成同一个前缀。这种情况下所有向量都在同一个坐标系里，看起来更「正常」，但查询与文档的对齐关系是错的，掉分同样查不出来。所以前缀应该封装在 embed 的调用约定里，而不是散在各处手拼。
  6. 可预期的追问：OpenAI 的模型要不要加前缀？不需要——它不属于这一族。所以这不是一条普遍规则，而是**每换一个模型都要重新读模型卡片确认**的事。

  How to reason about it · think before answering

  1. The core of this question is silent failure. Reciting 'e5 needs query: and passage: prefixes' is the baseline; explaining why nothing errors out and how you would catch it is what shows experience.
  2. The reason: these models are trained on pairs, short questions on one side and longer passages on the other, two genuinely different distributions. The prefix is a role marker learned during training. Omit it at inference and you are off-distribution.
  3. The consequence: the model still returns vectors, distances still compute, results still have an order, quality just degrades. Nothing throws, exactly like forgetting to normalise.
  4. How to catch it: run a small labelled question set against the same corpus twice, with and without prefixes, and compare hit rate. That is the evaluation gate built on day 8, and catching silent regressions is precisely what it is for.
  5. Mention the sneakier variant: prefixing at index time but not at query time, or using the same prefix on both sides. Everything sits in one coordinate space and looks healthier, yet the query-document alignment is wrong and the loss is just as invisible. Encapsulate prefixes in the embedding call convention rather than hand-writing them everywhere.
  6. Expected follow-up: do OpenAI models need prefixes? No, they are not in that family, so this is not a universal rule but a per-model detail you re-check on the model card every time you switch.

  答题要点

  • 这类模型用问句与段落的成对数据训练，前缀是区分两种角色的标记，缺了就落在训练分布之外。
  • 不加前缀不会报错，只会整体掉分，属于静默失效。
  • 唯一可靠的发现方式是拿一份标注问题集跑 A/B 对比命中率。
  • 更隐蔽的错法是两边前缀不一致或用了同一个前缀，看起来更正常但对齐是错的。
  • 前缀应封装在 embed 的调用约定里；换模型必须重读模型卡片，它不是普遍规则。

  Key points

  • These models are trained on question-passage pairs; the prefix marks which role a text plays, and omitting it puts you off-distribution.
  • Skipping prefixes never errors, it only degrades quality, so the failure is silent.
  • The reliable detection is an A/B run over a small labelled question set, comparing hit rate.
  • A subtler bug is mismatched or identical prefixes on both sides, which looks healthier but misaligns queries and documents.
  • Keep prefixes inside the embedding call convention, and re-read the model card whenever you switch models.
• 向量检索能完全取代关键词检索吗？举一个向量必然失手的查询，并说说你会怎么补。Can vector search fully replace keyword search? Give a query where vectors are bound to fail, and say how you would fix it.

  国内高频海外高频进阶#hybrid-search#embeddings#retrieval-failure

  分析过程 · 先想清楚再作答

  1. 这题是典型的「立场题」，答「能」或「不能」都不重要，重要的是你能不能举出一个具体到能复现的反例。举不出例子，前面说得再漂亮也会被判成没做过。
  2. 先给失手的类型，一次给全：错误码与状态码（429、E1032）、版本号与型号（v2.3.1、X20 Pro）、人名与工号、订单号与文档编号、以及否定表达。前四类的共同点是**这些词的价值在于字面唯一，而向量只保留语义邻近**，模型会把 429 和「限流」「超时」这些话题相近的东西编到一起，反而把真正写着 429 的那篇挤下去。
  3. 拿一个能复现的例子说：问「限流超了返回 429 吗」，BM25 稳稳命中写着 429 的接口文档，向量却可能把话题相近但没提 429 的产品手册排在前面。这个现象在本课第 2 天的实验里就能亲眼看到。
  4. 否定表达要单独强调：「支持导出 PDF」和「不支持导出 PDF」在向量空间里几乎重合，因为它们谈的是同一件事。指望向量区分肯定与否定一定翻车，这一层要靠生成侧读原文来判断。
  5. 怎么补：两路并行跑再融合，关键词一路用 BM25、向量一路用最近邻，用倒数排名融合把两个名次合成一个。这就是混合检索，本课第 9 天展开。要点是**两套的错法不一样**，所以合起来才有增益——如果两套错在同一批查询上，融合是白做的。
  6. 可预期的追问：那关键词一路能不能扔掉、改成让模型改写查询？可以缓解一部分（第 10 天的查询改写），但改写救不了字面唯一的标识符——你没法把 429 改写成别的说法。

  How to reason about it · think before answering

  1. This is a stance question where the stance matters less than the counter-example. Without a concrete, reproducible failing query, the rest of the answer reads as theory.
  2. Enumerate the failure classes up front: error and status codes, version numbers and SKUs, names and employee IDs, order or document identifiers, and negation. The first four share one property: their value lies in exact literal identity, which embeddings deliberately blur into semantic neighbourhoods.
  3. Give a reproducible example: ask whether rate limiting returns 429. BM25 lands on the API document that literally contains 429, while vector search may rank a topically similar product manual that never mentions the code.
  4. Call out negation separately: 'supports PDF export' and 'does not support PDF export' sit almost on top of each other because they discuss the same thing. Vectors cannot carry that distinction; the generation step reading the source has to.
  5. The fix: run both retrievers and fuse the rankings, BM25 on the lexical side and nearest neighbour on the vector side, combined with reciprocal rank fusion. That is hybrid search, covered on day 9. Fusion helps precisely because the two systems fail on different queries.
  6. Expected follow-up: could you drop the keyword path and rewrite queries instead? Rewriting helps with vocabulary mismatch, but it cannot rescue exact identifiers, since there is no paraphrase of 429.

  答题要点

  • 不能取代：错误码、版本号、人名、单号这类词的价值在于字面唯一，向量只保留语义邻近。
  • 具体反例：问「限流超了返回 429 吗」，BM25 命中写着 429 的文档，向量把话题相近却没提 429 的文档排前面。
  • 否定表达是另一类失手：肯定句与否定句在向量空间里几乎重合。
  • 补法是混合检索：两路并行再用倒数排名融合合并名次。
  • 融合有增益的前提是两套的错法不同；查询改写能缓解词汇不匹配，但救不了字面唯一的标识符。

  Key points

  • No: codes, version numbers, names and IDs matter as exact literals, which embeddings blur into neighbourhoods.
  • Concrete example: asking whether rate limiting returns 429, where BM25 hits the document containing 429 and vectors surface a topically similar one that never mentions it.
  • Negation is a second failure class, since affirmative and negative statements sit almost on top of each other.
  • The remedy is hybrid retrieval: run both paths and merge with reciprocal rank fusion.
  • Fusion pays off because the two paths fail differently; query rewriting helps vocabulary mismatch but not exact identifiers.

D3 文档进来这一关：PDF 与 HTML 解析、表格与扫描件、清洗规则和必须留下的元数据

• 一份 PDF 解析出来的文字顺序是乱的，你会怎么排查和修复？The text extracted from a PDF comes out in the wrong order. How do you diagnose and fix it?

  国内高频海外高频进阶#pdf-parsing#ingestion#data-quality

  分析过程 · 先想清楚再作答

  1. 这题在考你有没有真的动手解析过 PDF。区分度在第一句：能不能说出「PDF 里根本没有阅读顺序」这个前提。答不出这句的人，后面只会说「换个库试试」。
  2. 先给排查顺序：把抽出来的文本片段连同页码、坐标、字号一起打印出来，别只看拼好的字符串。乱序的原因几乎都藏在坐标里，看纯文本永远看不出来。
  3. 然后按现象分三类。左右两栏一行一行地交替，是多栏没识别；同一段话被拆成很多短片段且 y 值有回跳，是内容流按绘制顺序写的；文字整体没问题但夹着重复出现的短句，那不是乱序，是页眉页脚没剔。
  4. 修法对应着来：多栏就重建阅读顺序——把每页文字块的左边界排序找最大空隙当分栏线，再按「栏号、y 从大到小、x 从小到大」重排；页眉页脚按固定的 y 值带切掉，并打印剔除条数确认没误伤。
  5. 补一条能证明你在生产里干过的话：修完要有可回归的判据，不能靠肉眼。用乱序疑似度——顺着排好的顺序走一遍，统计「同栏内往回跳」和「从右栏跳回左栏」的比例，它不需要标准答案，可以挂进流水线天天跑。
  6. 可预期的追问：多栏识别错了怎么办？回答分两头——把分栏判定做保守（空隙不够宽、或者一侧内容占比太低就按单栏处理），并且让断言在双栏被误判成单栏时同样会报警，宁可漏修也不要悄悄改错。

  How to reason about it · think before answering

  1. This checks whether you have actually parsed a PDF yourself. The first sentence is the differentiator: a PDF has no reading order at all, only drawing instructions with coordinates.
  2. Start with the diagnostic step: dump the extracted fragments together with page, x, y and font size instead of looking at the concatenated string. The cause is always in the coordinates.
  3. Then classify the symptom. Lines alternating between left and right means multi-column layout was not detected. Fragments with y jumping backwards means the content stream was written in drawing order. Clean text sprinkled with a repeated short line is not disorder at all, it is a header or footer that was never stripped.
  4. Match the fix to the symptom. For columns, rebuild the order: sort the left edges of the fragments on each page, take the widest gap as the column boundary, then sort by column, then y descending, then x ascending. For headers and footers, cut fixed bands at the top and bottom and print how many fragments you dropped so you can confirm you did not cut into the body.
  5. Add the production-grade part: the fix needs a regression signal, not an eyeball check. Compute an out-of-order score by walking the sorted fragments and counting backward jumps within a column plus right-to-left column jumps. It needs no ground truth, so it can run on every ingest.
  6. Expected follow-up: what if column detection is wrong? Keep the detector conservative, treating a narrow gap or a lopsided split as single column, and make sure the assertion still fires when a two-column page is misread as one. Missing a fix is better than silently corrupting the order.

  答题要点

  • 前提先说清：PDF 只存「在某页某坐标画某段文字」，段落和阅读顺序都是解析时推出来的。
  • 排查时把片段连同页码、坐标、字号一起打印，纯文本看不出乱序的原因。
  • 三种典型成因：多栏没识别、内容流按绘制顺序写、页眉页脚没剔除。
  • 多栏的修法是找最大 x 空隙定分栏线，再按「栏号、y 降序、x 升序」重排。
  • 修完要有不依赖标准答案的回归指标，比如乱序疑似度，能挂进摄取流水线。

  Key points

  • State the premise: a PDF stores only drawing instructions, so paragraphs and reading order are inferred, not read.
  • Debug by dumping fragments with page, coordinates and font size; plain text hides the cause.
  • Three common causes: undetected multi-column layout, content stream written in drawing order, and headers or footers left in.
  • Fix columns by finding the widest gap between left edges and sorting by column, then y descending, then x ascending.
  • Add a ground-truth-free regression metric such as an out-of-order score so the fix stays fixed.
• 文档解析阶段应该保留哪些元数据？少了其中某一项会在哪个环节出问题？Which metadata should a document parsing stage preserve, and which downstream feature breaks if you drop each one?

  国内高频海外高频进阶#metadata#ingestion#access-control

  分析过程 · 先想清楚再作答

  1. 这题最容易答成列清单。区分度不在你能列出几个字段，而在能不能给每个字段配一个具体的下游功能——列了八个字段却说不出谁在用，等于没设计过。
  2. 用一条判据把字段选出来：删掉之后还能不能从原件重新恢复。不能恢复的，解析时就必须留；能恢复的（比如格式、空白）可以放心丢。
  3. 然后一一对应地说：块编号支撑可验证的引用，没有它引用就只能靠模型自觉；标题路径支撑「这句话出自哪一节」和按结构切块；页码支撑引用精确到页；权限标签支撑检索层过滤；更新时间支撑材料冲突时的取舍；内容指纹支撑增量同步。
  4. 挑两个讲透代价。权限标签少了，等到要做访问控制时只能全量重新解析一遍；更糟的是有人会图省事在生成阶段过滤，那等于内容已经进了上下文，泄露已经发生。
  5. 内容指纹少了，每次同步都是全量重建：重新解析、重新切块、重新向量化。一份几千篇的知识库每天重算一次，光 embedding 的账单就够说服任何人。
  6. 可预期的追问：字段拿不准要不要留怎么办？答保守——存储是整条链路上最便宜的一环，加一个字段的代价远小于重跑一次全量解析。

  How to reason about it · think before answering

  1. The trap here is answering with a bare list. The differentiator is pairing every field with a concrete downstream feature. Listing eight fields without naming who consumes them shows you never designed one.
  2. Give the selection rule first: can this be recovered from the original file later? If not, it must be captured at parse time. Formatting and whitespace can be dropped because the original still has them.
  3. Then map fields to consumers: a stable chunk id makes citations verifiable, a heading path tells the user which section a sentence came from and enables structure-aware chunking, page numbers make citations land on the right page, an access-control label enables filtering inside retrieval, an updated-at date resolves conflicting sources, and a content hash enables incremental sync.
  4. Take two of them all the way to cost. Without the access label you must re-parse the whole corpus when access control lands, and worse, people work around it by filtering at generation time, which means the content already reached the context and the leak already happened.
  5. Without a content hash, every sync is a full rebuild: re-parse, re-chunk, re-embed. For a few thousand documents synced daily, the embedding bill alone settles the argument.
  6. Expected follow-up: what about a field you are unsure of? Be conservative. Storage is the cheapest part of the pipeline, and adding a field costs far less than re-running a full parse.

  答题要点

  • 判据是「删了还能不能从原件恢复」，不能恢复的必须在解析时留下。
  • 块编号服务于可验证的引用，标题路径服务于定位与按结构切块，页码服务于引用精确到页。
  • 权限标签必须在解析时打上，否则做访问控制时要全量重解析，且容易被错误地放到生成阶段过滤。
  • 更新时间用于材料冲突时并列两种说法，内容指纹用于增量同步，少了它每次都要全量重建。
  • 拿不准就保守保留：加一个字段的成本远低于重跑一次全量解析。

  Key points

  • The rule is recoverability: if it cannot be recovered from the original later, capture it at parse time.
  • Chunk ids back verifiable citations, heading paths back localisation and structure-aware chunking, page numbers make citations land precisely.
  • Access-control labels must be attached during parsing, otherwise enabling ACL means re-parsing everything, and teams end up filtering at generation time where the leak has already occurred.
  • Updated-at lets you present conflicting sources side by side; a content hash enables incremental sync instead of full rebuilds.
  • When unsure, keep the field: storage is far cheaper than a full re-parse.

D4 切块策略：固定、递归、按结构、父子与语义五种切法，以及用评估而不是直觉来选

• 你怎么决定切块大小？说出你会看的两个指标和一个反例。How do you decide on chunk size? Name two metrics you would look at, and one counterexample.

  国内高频海外高频进阶#chunking#evaluation

  分析过程 · 先想清楚再作答

  1. 这题的题眼是「怎么决定」，不是「多大合适」。答一个具体数字（512 token、1000 字符）就已经输了——面试官想看的是你有没有一套定法，而不是你记得住哪个默认值。
  2. 先把矛盾摆出来：块大则信噪比低、上下文贵，块小则单块缺语境、模型答不出所以然。切块大小就是在这两头之间找位置，所以两个指标必须分别对应这两头。
  3. 第一个指标是检索侧的命中率——答案文档有没有进上下文。第二个是生成侧的可用性，最省事的代理指标是切碎率，也就是有多少块结尾停在半句话上；再往前一步就是忠实度和引用是否可定位。
  4. 关键补一句：两个指标必须在**同一个 token 预算**下比，不能按「取前 k 块」比。k 固定时块越大塞进去的字越多，大块切法会赢在买得多而不是切得准上。这一句往往是这道题的区分点。
  5. 反例要具体。最好用的一个是：把块从 400 字调到 1200 字，命中率不降反升——但那是因为一整篇短文档被当成一块塞了进去，检索其实什么都没做，等于退化成了全文投喂。指标涨了，系统更差了。
  6. 可预期的追问是「那你第一次上手时从哪个数字起步」。答：先按文档类型选切法（有标题层级就按结构切），块长从 300 到 500 字起步、重叠取一到两成，然后立刻建一组标准问题跑评估，用两三轮迭代把它调到位。起步值是起步值，不是结论。

  How to reason about it · think before answering

  1. The question is about method, not about a number. Answering with a specific default (512 tokens, 1000 characters) already loses it — the interviewer wants to hear that you have a procedure.
  2. State the tension first: large chunks dilute the signal and cost context; small chunks lose the surrounding meaning so the model cannot use them. The two metrics you name should map onto those two failure modes.
  3. Metric one is retrieval-side hit rate: did a document that actually answers the question make it into the context. Metric two is generation-side usability, cheaply proxied by the fraction of chunks that end mid-sentence, and more seriously by faithfulness and whether citations resolve.
  4. Add the point that separates candidates: both metrics must be compared under the same token budget, never under a fixed top-k. With fixed k, bigger chunks simply buy more text and win for the wrong reason.
  5. Make the counterexample concrete: raising chunk size from 400 to 1200 characters can lift hit rate purely because whole short documents now fit in one chunk, which means retrieval stopped doing anything and you are back to stuffing full documents. The metric improved while the system got worse.
  6. Expect the follow-up: where do you start on day one. Pick the strategy from the document type first (structural splitting whenever headings exist), start around 300 to 500 characters with 10 to 20 percent overlap, then build a golden set immediately and iterate. A starting point is not a conclusion.

  答题要点

  • 先按文档类型选切法，再调长度：有标题层级就按结构切，没有结构才谈固定长度或语义。
  • 看两个指标：检索侧的命中率，生成侧的切碎率（进一步是忠实度与引用可定位性）。
  • 两个指标必须在同一个 token 预算下比，不能按「取前 k 块」比，否则大块只是买得更多。
  • 反例：块调大后命中率上升，但那是因为整篇被当成一块，检索退化成全文投喂。
  • 起步值 300 到 500 字、重叠一到两成，然后靠一组固定问题迭代，不靠直觉定稿。

  Key points

  • Choose the strategy from the document type first, then tune length: split on headings whenever the structure survives parsing.
  • Watch two metrics: retrieval hit rate on one side, mid-sentence break rate (then faithfulness and citation resolvability) on the other.
  • Compare under an equal token budget, never a fixed top-k, or larger chunks win by buying more text.
  • Counterexample: hit rate rises after enlarging chunks because whole documents now fit in one chunk and retrieval has effectively stopped working.
  • Start near 300 to 500 characters with 10 to 20 percent overlap, then iterate against a fixed question set instead of guessing.
• 父子切块的收益是什么？它在什么情况下反而会拖慢系统？What does parent-child chunking buy you, and when does it slow the system down instead?

  国内高频海外高频进阶#chunking#parent-child

  分析过程 · 先想清楚再作答

  1. 这题考的是你有没有意识到「检索单位」和「上下文单位」可以是两个东西。答不出这句话，后面说什么都是复述。
  2. 收益一句话说清：小块进索引，信噪比高、容易被找到；命中之后顺着父指针把整节回填给模型，语境完整。精度和完整度这次不用二选一。
  3. 拖慢的场景要从代价一条条推。第一条是上下文预算：每命中一个新子块可能拖进来一整个父节，同样的 token 预算装不下几条，检索结果的多样性反而变差。
  4. 第二条是写入侧：父子两套都要维护，文档更新时两边都要重算，块 id 的稳定性也更难保证，增量同步的复杂度明显上升。
  5. 第三条是收益消失的条件：当文档本身的小节就不长时，父块和子块差不多大，你付了两套索引的钱，什么也没多买到。所以父子切块适合长节、深层级的文档，不适合结构本来就细碎的知识库。
  6. 可预期的追问是「那和直接把块切大有什么区别」。答：切大是把噪声一起放进索引，父子是只把噪声放进上下文、不放进索引——被检索的那一段始终是干净的短文本，这是本质区别。

  How to reason about it · think before answering

  1. This question checks whether you know that the retrieval unit and the context unit can be two different things. Without that sentence, everything else is recitation.
  2. State the benefit compactly: small chunks go into the index so they are easy to match, and once a child is hit you follow the parent pointer and hand the model the whole section. You stop trading precision against completeness.
  3. Derive the slowdown from the costs. First, the context budget: every new child may drag in an entire parent, so an equal budget holds fewer distinct pieces and result diversity drops.
  4. Second, the write path: two levels to maintain, both recomputed on every document update, and chunk ids become harder to keep stable, which makes incremental sync noticeably more complex.
  5. Third, the condition under which the benefit disappears: when sections are already short, the parent and the child are nearly the same text, so you paid for two indexes and bought nothing. Parent-child suits long sections and deep hierarchies, not already fine-grained knowledge bases.
  6. Expect the follow-up: how is this different from simply using bigger chunks. Bigger chunks put the noise into the index; parent-child puts the noise only into the context. What gets matched stays short and clean.

  答题要点

  • 核心是把检索单位和上下文单位拆开：小块负责被找到，大块负责被读懂。
  • 收益是精度与完整度同时拿到，不用在信噪比和语境之间二选一。
  • 代价一：一次命中可能拖进整个父节，同样的上下文预算装得下的条数变少，结果多样性下降。
  • 代价二：父子两套索引都要维护与重算，文档更新时增量同步的复杂度明显上升。
  • 失效场景：文档小节本来就短时父子块差不多大，多付一套成本却没多买到东西。

  Key points

  • The core idea is decoupling the retrieval unit from the context unit: small chunks get found, large chunks get understood.
  • The payoff is precision and completeness at the same time instead of trading one for the other.
  • Cost one: a single hit can drag in a whole parent, so an equal context budget holds fewer distinct results and diversity suffers.
  • Cost two: two index levels to maintain and recompute, which makes incremental sync on document updates considerably harder.
  • It stops paying off when sections are already short, because parent and child are nearly identical and you bought nothing for the extra cost.

D5 向量索引与库选型：HNSW 与倒排文件、量化省内存、带过滤的查询与多租户隔离

• 分层可导航小世界图和倒排文件索引你会怎么选？各说一个必须选它的场景，以及各自最该调的参数。How do you choose between an HNSW index and an IVFFlat index? Give one scenario that forces each choice, and name the parameter you would tune first in each.

  国内高频海外高频进阶#vector-index#hnsw#ivfflat

  分析过程 · 先想清楚再作答

  1. 这题的区分度不在能不能背出两种结构，而在你会不会给出触发条件。只说「HNSW 快、IVFFlat 省内存」的人一抓一大把，面试官等的是「什么情况下我必须选另一个」。
  2. 先用两句话把结构说清：HNSW 是分层的邻居图，查询从稀疏的上层跳到稠密的下层，逐步逼近；IVFFlat 是先聚类成若干个列表，查询时只在最近的几个列表里扫。一个是图上导航，一个是分区搜索。
  3. 再把参数对应上去：HNSW 建图有 m 与 ef_construction，查询有 ef_search；IVFFlat 建索引有 lists，查询有 probes。**先调查询侧参数**，因为它不用重建索引、能逐次查询调整，是唯一一个上线之后还能动的旋钮。
  4. 给两个反向的必须场景：数据分钟级高频写入、且内存和建索引窗口都紧张时必须选 IVFFlat，因为 HNSW 的图会持续膨胀、重建代价高；反过来，数据相对静态、查询延迟有硬性 SLA 时必须选 HNSW，因为同等召回下它的延迟更低。
  5. 补一条容易被忽略的工程细节：IVFFlat 的聚类是建索引那一刻的数据决定的，数据分布漂移之后召回会悄悄下滑，所以它需要一条定期重建的运维流程；HNSW 没有这个包袱，但它的索引往往比表本身还大。
  6. 可预期的追问：probes 和 ef_search 的默认值分别是多少？答 1 和 40，并且要主动说出 IVFFlat 默认 probes = 1 意味着只看一个列表，建完索引不设 probes 基本等于没调过——这是新手最常见的事故。

  How to reason about it · think before answering

  1. The differentiator is not describing both structures, it is naming the condition that forces one over the other. Saying 'HNSW is faster, IVFFlat is cheaper' is what everyone says.
  2. Describe the structures in one line each: HNSW is a layered neighbour graph you navigate from sparse upper layers down to dense lower ones; IVFFlat clusters vectors into lists and only scans the lists closest to the query.
  3. Map the knobs: HNSW builds with m and ef_construction and queries with ef_search; IVFFlat builds with lists and queries with probes. Tune the query-side knob first, because it needs no rebuild and is the only one you can still move after launch.
  4. Give two forcing scenarios in opposite directions. Minute-level write traffic with tight memory and a short build window forces IVFFlat, since an HNSW graph keeps growing and is expensive to rebuild. A largely static corpus with a hard latency SLA forces HNSW, since it hits the same recall at lower latency.
  5. Add the operational detail people forget: IVFFlat clusters reflect the data at build time, so recall degrades silently as the distribution drifts and you need a scheduled rebuild. HNSW avoids that but its index is often larger than the table.
  6. Expected follow-up: what are the defaults? probes is 1 and ef_search is 40. Volunteer that leaving probes at 1 means scanning a single list, which is the single most common IVFFlat mistake.

  答题要点

  • HNSW 是分层邻居图，IVFFlat 是先聚类再局部扫描；前者查询质量优先，后者建索引与内存开销优先。
  • 先调查询侧参数：HNSW 调 ef_search，IVFFlat 调 probes，两者都不需要重建索引。
  • 高频写入、内存与建索引窗口紧张选 IVFFlat；数据相对静态、延迟有硬性要求选 HNSW。
  • IVFFlat 的聚类会随数据漂移失真，需要定期重建；HNSW 没这个问题但索引常常比表还大。
  • 默认值要记住：probes 是 1、ef_search 是 40，建完索引不调 probes 等于没用上索引的能力。

  Key points

  • HNSW is a layered neighbour graph; IVFFlat clusters first and scans a subset of lists. HNSW favours query quality, IVFFlat favours build cost and memory.
  • Tune the query-side knob first: ef_search for HNSW, probes for IVFFlat. Neither needs a rebuild.
  • Heavy write traffic with tight memory and build windows points to IVFFlat; a static corpus with a hard latency SLA points to HNSW.
  • IVFFlat clusters drift with the data and need scheduled rebuilds; HNSW does not, but its index is often larger than the table.
  • Know the defaults: probes 1, ef_search 40. Leaving probes at 1 wastes the index.
• 把向量从全精度换成半精度或二值量化，你会用什么方法确认召回没有明显下降？If you switch your vectors from full precision to half precision or binary quantisation, how do you verify that recall has not dropped materially?

  国内高频海外高频进阶#quantization#evaluation#recall

  分析过程 · 先想清楚再作答

  1. 这题表面问量化，实际问的是你会不会做评估。只回答「跑几个问题看看结果对不对」的人会被直接判为没做过——面试官想听的是一套可复现的量法。
  2. 先把真值这件事说死：真值必须来自暴力全量比对，也就是把索引关掉、全表算距离取前 k。拿索引结果当真值是最常见的自欺，因为那样量出来的召回永远接近 100%，你会以为量化无损。
  3. 然后给流程：固定一批查询（几十条起步，覆盖长短查询和不同主题），先用全精度算出真值，再换量化重跑，计算召回率@k。同时记录三件事——索引大小、建索引耗时、查询延迟的中位数与 p95，只报召回是不够的。
  4. 补一条判据：量化损失有多大取决于向量分布，别人的数字不能抄。稀疏向量对二值量化尤其不友好，因为二值化只保留符号位，零和负数会被压成同一个值，信息几乎被抹平。所以换方案必须在自己的数据上重新量一次。
  5. 结论要给可操作的建议：半精度通常近乎无损，还能把建索引维度上限从 2000 提到 4000，是默认可以先上的一档；二值量化损失明显，标准用法是拿它粗筛一批候选，再用原始向量在这一小批里精排，粗筛窗口越宽召回补得越多、延迟也越高。
  6. 可预期的追问：召回掉了多少算可以接受？答这取决于下游——后面还有重排时，粗排召回掉两三个点通常无感；如果检索结果直接进提示词，掉一个点就意味着每一百次回答里多一次缺材料。要把这个判断挂到业务指标上，而不是拍一个阈值。

  How to reason about it · think before answering

  1. The question looks like it is about quantisation, but it is really about whether you know how to evaluate. Answering 'try a few queries and eyeball it' fails immediately.
  2. Pin down ground truth first: it must come from an exhaustive scan with the index disabled. Using index results as ground truth is the classic self-deception, because recall then looks close to 100% no matter what you changed.
  3. Give the procedure: fix a query set of at least a few dozen covering short and long queries across topics, compute ground truth at full precision, rerun with the quantised representation, and report recall at k. Report index size, build time, and median plus p95 latency alongside it, because recall alone is not a decision.
  4. Add the judgement rule: quantisation loss depends on your vector distribution, so published numbers do not transfer. Sparse vectors suffer badly under binary quantisation because only the sign bit survives and zeros collapse together.
  5. Land on something actionable: half precision is usually near lossless and raises the indexable dimension ceiling from 2000 to 4000, so it is a safe first step. Binary quantisation loses real recall and should be used as a cheap first pass, re-ranked with the original vectors over a wider candidate window.
  6. Expected follow-up: how much loss is acceptable? It depends on what comes next. With a re-ranker downstream, a couple of points off first-stage recall is usually invisible; if retrieval feeds the prompt directly, one point means one more unanswerable question per hundred. Tie the threshold to a product metric, not to a number you made up.

  答题要点

  • 真值必须来自关掉索引的暴力全量比对，拿索引结果当真值会让召回永远接近 100%。
  • 固定一批查询，量化前后跑同一批，报召回率@k，同时报索引大小、建索引耗时和延迟分位数。
  • 量化损失取决于向量分布，别人的数字不能抄，必须在自己的数据上重新量。
  • 半精度通常近乎无损，还能把索引维度上限从 2000 提到 4000，可以作为默认第一档。
  • 二值量化损失明显，正确用法是粗筛加原始向量重排，粗筛窗口越宽召回补得越多、延迟越高。

  Key points

  • Ground truth must come from an exhaustive scan with indexes disabled; using index output as truth pins recall near 100%.
  • Run one fixed query set before and after, report recall at k together with index size, build time and latency percentiles.
  • Quantisation loss depends on your own vector distribution, so measure it on your data instead of quoting benchmarks.
  • Half precision is usually near lossless and raises the indexable dimension limit from 2000 to 4000, making it a safe default.
  • Binary quantisation loses real recall; use it as a cheap first pass and re-rank with the original vectors over a wider window.
• 什么时候应该把向量搬出 PostgreSQL？给出可量化的触发条件，也说说不该搬的理由。When should you move your vectors out of PostgreSQL into a dedicated vector database? Give measurable triggers, and also make the case for staying.

  国内高频海外高频进阶#vector-database#architecture#trade-offs

  分析过程 · 先想清楚再作答

  1. 这题考的是工程判断，不是技术偏好。开口就说「专用向量库更专业」的人会被追问到答不上来；面试官想看的是你有没有把迁移成本算进去。
  2. 先给默认立场并给出理由：第一版留在 PostgreSQL，因为事务、备份、时间点恢复、权限、跟业务表 JOIN 和现成的运维工具全是白送的。多一个数据库就多一份同步、一份一致性问题、一份值班负担，这些成本很少被写进选型文档。
  3. 然后给四条可量化的触发线：数据量（判据不是行数而是索引还塞不塞得进内存）、写入频率（分钟级流式更新会让聚类失真、让图持续膨胀）、过滤复杂度（十几个属性的任意组合让部分索引和分区都排列组合不过来）、团队运维能力（没人愿意长期照看第二个数据库，前三条再成立也别搬）。
  4. 第三条要展开一点，因为它最常是真正的原因：专用向量库把过滤做进了索引结构本身，而不是扫完索引再筛，所以在复杂过滤下天然占优。把这一点说出来，说明你理解的是机制而不是口碑。
  5. 还要主动给一条常被忽略的替代路径：很多「向量检索不够用」的问题，真正的解法是混合检索加重排，而不是换数据库。先把关键词一路加回来、把重排接上，再决定要不要搬——顺序搞反了会白搬一次。
  6. 可预期的追问：真要搬怎么迁？答分三步——先双写并在影子流量上比对两边的召回与延迟，再把读流量按比例切过去，最后才停掉旧路径。中间任何一步指标不达标就停下，这比一次性切换安全得多。

  How to reason about it · think before answering

  1. This tests engineering judgement, not tooling preference. Opening with 'dedicated vector databases are better' invites follow-ups you cannot answer.
  2. State the default position and justify it: keep the first version in PostgreSQL, because transactions, backups, point-in-time recovery, permissions, joins with business tables and the tooling your team already knows all come free. A second datastore adds synchronisation, a consistency surface and an on-call burden that selection documents rarely price in.
  3. Then give four measurable triggers: data volume (the test is whether the index still fits in memory, not the raw row count), write frequency (minute-level streaming updates distort clusters and inflate graphs), filter complexity (arbitrary combinations of a dozen attributes defeat both partial indexes and partitioning), and operational capacity.
  4. Expand on filter complexity, because it is most often the real reason: dedicated vector databases push filtering into the index structure instead of applying it after the scan, which is a mechanical advantage rather than a reputational one.
  5. Volunteer the alternative people skip: many 'vector search is not good enough' problems are actually solved by hybrid retrieval plus re-ranking, not by a new database. Add the keyword path and a re-ranker first, then decide.
  6. Expected follow-up: how would you migrate? Dual-write, compare recall and latency on shadow traffic, shift read traffic gradually, and only then retire the old path. Stop at any step where the metrics regress.

  答题要点

  • 默认留在 PostgreSQL：事务、备份、恢复、权限、JOIN 和现成运维都是白送的，多一个库就多一份同步与值班成本。
  • 触发线一是数据量，判据是索引还塞不塞得进内存，而不是行数本身。
  • 触发线二是写入频率，分钟级流式更新会让聚类失真、让图持续膨胀。
  • 触发线三是过滤复杂度，专用库把过滤做进索引结构，复杂过滤下有机制上的优势。
  • 触发线四反过来看：没有长期运维第二个数据库的人手，前三条成立也不该搬；很多问题的真正解法是混合检索加重排。

  Key points

  • Default to staying in PostgreSQL: transactions, backups, recovery, permissions, joins and familiar tooling are free, and a second store adds sync and on-call cost.
  • Trigger one is data volume, measured by whether the index still fits in memory rather than by row count.
  • Trigger two is write frequency: minute-level streaming updates distort clusters and inflate graphs.
  • Trigger three is filter complexity: dedicated stores push filtering into the index structure, a mechanical advantage under complex predicates.
  • Trigger four cuts the other way: without people to run a second database, do not move even if the first three hold. Often hybrid retrieval plus re-ranking is the real fix.

D6 生成这一侧：上下文怎么排、引用怎么标、什么时候必须拒答，以及流式回答

• 怎么让模型的引用是真的而不是编的？说出一个不依赖模型自觉的方案。How do you make sure a model's citations are real rather than fabricated? Describe a scheme that does not rely on the model behaving well.

  国内高频海外高频进阶#citation-verification#grounding#hallucination

  分析过程 · 先想清楚再作答

  1. 题眼在「不依赖模型自觉」这半句。回答里只要出现「在提示词里强调请确保引用准确」，这题就答砸了——面试官问的正是提示词管不住的那部分。
  2. 先把问题拆成两半：引用要能验证，前提是它是一个**闭集里的符号**，不是一段自由文本。所以第一步是组装上下文时给每块材料一个编号，提示词里明确只能引用发出去的编号。让模型写「根据《某某手册》」是没法验证的，标题是它可以随口生成的字符串。
  3. 第二步是事后核对，两道闸缺一不可。第一道查编号存在性：发出去的是 1 到 5，出现 8 就一定是编的，一行代码判掉。第二道查实质重合：编号是真的、内容却对不上，这类更隐蔽，要算这句话的词元有多大比例能在被引块原文里找到，低于阈值判不通过。
  4. 算重合度时有个坑要主动说出来：先剔掉在多数块里都出现的高频词元，否则「文件」「系统」这种词会让随便哪一块都及格。这跟 BM25 用逆文档频率压常见词是同一个道理。
  5. 校验不过怎么办：把具体原因写成反馈打回去重生成一次，只给一次机会；连着两版都编说明材料本来就不支持，该走拒答而不是第三次重试。另外校验必须拿原文比对，不能拿压缩或改写过的材料比对，否则「校验通过」保证不了用户点开看到的东西。
  6. 可预期的追问：为什么不让模型自己再检查一遍？因为自检和生成是同一个模型的同一种倾向，它对自己编的东西没有独立信息源；而编号核对是一个确定性判断，成本几乎为零、结果可复现，这两点自检都做不到。

  How to reason about it · think before answering

  1. The phrase to catch is 'not relying on the model behaving well'. Any answer that boils down to 'tell the model to be accurate in the prompt' fails, because the prompt is exactly the part that cannot enforce this.
  2. Split the problem in two. Verifiability requires that a citation be a symbol from a closed set, not free text. So step one is numbering the blocks at assembly time and telling the model it may only cite the numbers it was given. 'According to the storage handbook' cannot be checked, because the title is a string the model can invent.
  3. Step two is post-hoc checking, with two gates. Gate one is existence: you handed out 1 through 5, so an 8 is fabricated, and that is a one-line check. Gate two is substantive overlap, which catches the sneakier case where the number is real but the block says something else. Measure what fraction of the sentence's terms appear in the cited block and reject below a threshold.
  4. Mention the trap in the overlap metric: drop terms that appear in most blocks first, otherwise generic words let any citation pass. It is the same reasoning behind inverse document frequency in BM25.
  5. On failure, feed the specific reason back and regenerate once, not repeatedly. Two fabricated drafts in a row means the material does not support the question, so refuse instead. Also verify against the original chunk text, never against a compressed or rewritten version, otherwise 'verified' says nothing about what the user sees.
  6. Expected follow-up: why not ask the model to self-check? Self-checking shares the generator's bias and has no independent source of truth, whereas number checking is deterministic, essentially free, and reproducible.

  答题要点

  • 引用必须是块编号这种闭集符号，不能是自由文本的文档标题——可验证性来自闭集，不来自措辞。
  • 两道闸：编号存在性，以及这句话与被引块原文的实质重合度，后者才拦得住「编号是真的、内容对不上」。
  • 算重合度前剔掉在多数块里都出现的高频词元，否则随便引哪一块都能及格。
  • 校验不过就带着具体原因打回重生成一次，只给一次机会，两版都编就转拒答。
  • 校验对象必须是用户能点开看到的原文，不是压缩或改写后的材料。

  Key points

  • Citations must be closed-set symbols such as block numbers, not free-text titles: verifiability comes from the closed set, not from wording.
  • Two gates: the number must exist, and the sentence must substantively overlap the cited block's original text, which is what catches real-number-wrong-content fabrication.
  • Strip terms that occur in most blocks before scoring overlap, or any citation will pass.
  • On failure, regenerate once with the concrete reason fed back; two bad drafts means refuse instead.
  • Always verify against the original text the user can open, never against a compressed or rewritten copy.

D7 第一周综合：把六天的零件装成一个可一键启动的检索问答服务并复盘

• 摄取链路和查询链路应该共享哪些代码？强行复用会带来什么具体问题？What should the ingestion path and the query path share, and what concretely goes wrong when you over-share?

  国内高频海外高频进阶#architecture#ingestion#retrieval

  分析过程 · 先想清楚再作答

  1. 题眼在「强行」两个字。面试官想看的是你能不能说出复用的边界，而不是背诵「不要重复自己」。
  2. 先说清两条链路的性质差异：摄取是批处理，几十秒跑完，失败重跑一遍就行；查询是在线请求，几百毫秒要出结果，失败用户当场看到。错误处理、超时、并发策略天然不同。
  3. 所以结论是：**共享接口，不共享流程**。两边唯一该共享的是存储层的那个接口，以及 embedding 的函数签名——注意后者共享的是签名和模型选择，不是调用流程。
  4. 给出强行复用的具体症状：抽出来的公共模块里开始出现 isIngest 这类分支，一个改动要同时验证两条链路，最后没人敢动它。
  5. 补一条真正必须一致的东西：给块算向量和给问题算向量必须用同一个模型。这不是复用代码，是复用配置——而且要把模型名写进向量表，否则模型换了没人发现，检索会静默地返回垃圾。
  6. 可预期的追问：那切块逻辑呢？查询侧压根不切块，所以它只属于摄取链路；真要在查询侧用到（比如 D11 的父子回填），走的也是存储层读回大块，不是把切块器搬过来。

  How to reason about it · think before answering

  1. The word to notice is 'over-share'. The interviewer wants the boundary, not a recital of DRY.
  2. Start from how the two paths differ. Ingestion is batch: tens of seconds, and a failure just means rerunning it. Query is online: hundreds of milliseconds, and a failure is visible to the user immediately. Error handling, timeouts and concurrency are simply not the same problem.
  3. Hence the rule: share the interface, not the flow. The only genuinely shared thing is the storage interface, plus the embedding function signature.
  4. Name the symptom of over-sharing: the extracted module fills up with isIngest branches, every change has to be verified on both paths, and eventually nobody dares touch it.
  5. Add the one thing that truly must match: chunks and queries must be embedded by the same model. That is shared configuration, not shared code, and the model name belongs in the vector table so a silent mismatch is detectable.
  6. Expected follow-up: what about chunking? The query path never chunks. Even when it needs a parent block, it reads it back through storage rather than importing the chunker.

  答题要点

  • 共享接口不共享流程：唯一的交界是存储层，加上 embedding 的函数签名。
  • 两条链路的错误处理与延迟约束根本不同，批处理可以重跑，在线请求必须快速失败。
  • 强行复用的症状是公共模块里长出 isIngest 分支，改一次要验两条链路。
  • 必须一致的是模型选择而不是代码：块与查询要用同一个 embedding 模型，并把模型名记进向量表。
  • 切块只属于摄取；查询侧需要大块时通过存储层读回，而不是把切块器搬过去。

  Key points

  • Share the interface, not the flow: storage is the only boundary, plus the embedding signature.
  • The two paths have different error handling and latency budgets; batch can rerun, online must fail fast.
  • Over-sharing shows up as isIngest branches and changes that must be verified twice.
  • What must match is the model choice, not the code: record the model name alongside every stored vector.
  • Chunking belongs to ingestion only; the query path reads larger units back through storage.
• 一个检索问答服务上线前你会做哪三项检查？为什么偏偏是这三项？What three checks would you run before shipping a retrieval QA service, and why those three?

  国内高频海外高频进阶#production-readiness#citations#refusal

  分析过程 · 先想清楚再作答

  1. 这题的区分度不在你能列几项，而在你能不能说清「为什么是这三项」。列十项而每项都不给理由，反而说明你没有排过优先级。
  2. 推导方式是按后果排序：哪种故障用户看不出来、又损失最大，哪一项就该排在前面。
  3. 第一项是引用可查证：每条引用的编号都能回查到真实存在的块，且那一块确实与该句有实质重合。这一项排第一是因为引用错了用户根本发现不了，而它恰恰是这类系统唯一的信任来源。
  4. 第二项是该拒答时真的拒答：构造一个语料里没有答案的问题，看它是回那句拒答话术还是开始编。这一项也属于用户看不出来的故障，且一旦编造被发现，整个系统的可信度归零。
  5. 第三项是摄取到检索的一致性：摄取完之后新文档立刻能被检索到，且关键词与向量两路的覆盖数量对得上。这一项防的是「一路能查一路查不到」这种最难排查的故障。
  6. 可预期的追问：为什么延迟和成本不在前三？因为它们是**看得见**的故障——慢了用户会抱怨，贵了账单会告诉你；而上面三项不检查就永远不会有人告诉你。

  How to reason about it · think before answering

  1. The discriminator is not how many checks you list but whether you can justify the three. Ten items with no ranking suggests you have never had to prioritise.
  2. Derive them by consequence: the failures that are invisible to users and most damaging go first.
  3. First, citations must be verifiable: every cited id resolves to a real chunk, and that chunk genuinely overlaps the sentence citing it. This ranks first because a wrong citation is undetectable by the user, and citations are the only source of trust this system has.
  4. Second, refusal must actually fire: ask a question the corpus cannot answer and confirm the system says so instead of inventing. Also invisible, and one discovered fabrication zeroes out trust in the whole product.
  5. Third, ingestion-to-retrieval consistency: freshly ingested documents are retrievable immediately, and the keyword and vector paths cover the same set. This guards against the 'one route finds it, the other does not' failure, which is the hardest to diagnose.
  6. Expected follow-up: why not latency and cost? Because those failures are visible. Users complain about slowness and the bill reports overspending; nobody will ever report the three above.

  答题要点

  • 先给排序依据：优先检查用户发现不了、但后果最重的故障。
  • 第一项引用可查证：编号能回查到真实的块，且该块与被引的那句话有实质重合。
  • 第二项拒答生效：用一个语料里没有答案的问题验证系统会说查不到，而不是开始编。
  • 第三项摄取与检索一致：新入库的文档立刻可检索，关键词与向量两路覆盖对得上。
  • 延迟和成本重要但排在后面，因为它们是看得见的故障，会自己找上门。

  Key points

  • State the ranking rule first: prioritise failures users cannot see but that cost the most.
  • Check one, verifiable citations: every id resolves to a real chunk that overlaps the sentence citing it.
  • Check two, refusal actually fires on a question the corpus cannot answer.
  • Check three, ingestion and retrieval agree: new documents are immediately retrievable on both routes.
  • Latency and cost matter but rank lower because those failures announce themselves.

D8 评估先行：搭 golden set、算召回与排序指标、用模型当裁判判忠实度

• 让你从零给一个公司知识库的 RAG 系统建评估集，你会怎么做？多少题才算够用？You need to build an evaluation set from scratch for a RAG system over a company knowledge base. How would you do it, and how many questions are enough?

  国内高频海外高频进阶#evaluation#golden-set#rag

  分析过程 · 先想清楚再作答

  1. 这题的区分度在「出题方向」和「规模的理由」两处。开口就说「找几百个用户真实问题」的，多半没真做过——真实问题的答案在哪篇文档里，没人标得出来。
  2. 先给方向：从语料反向出题，打开每一篇读它能回答什么，出题的那一刻答案文档就已经确定了，标注成本几乎为零。反方向（先想问题再找答案）会得到一堆自己都不知道答案的题。
  3. 再给结构：每题记问题、答案文档列表、类型三个字段；类型至少分单文档、多跳、无答案三类，并说明多跳必须全部答案文档命中才算命中，无答案不参与召回率而是考拒答。
  4. 规模的理由要给出来，不能只报一个数字：20 题能把「完全不能用」和「基本能用」分开，够做冒烟；100 到 200 题才有资格判断「涨了两个点」是真的还是噪声。上线之后每次线上出问题就把那个问题补进集合——评估集是长出来的。
  5. 补一句成本与保鲜：出题是人力活，20 题两小时是正常量级；语料更新后要复核答案文档还在不在，否则集合会悄悄腐烂，指标下跌你会误以为是系统坏了。
  6. 可预期的追问是「怎么防止评估集被过拟合」。答案是留一份不参与调优的保留集，并且定期从线上真实问题里补充新题，只用来验收不用来调参。

  How to reason about it · think before answering

  1. The discriminator here is the direction you generate questions in, and whether you can justify a size rather than name one.
  2. Go corpus-first: read each document and write the questions it can answer. The answer document is fixed at authoring time, so labeling is nearly free. Question-first gives you items whose answers nobody can locate.
  3. Give the schema: question, answer document ids, and a type. At minimum three types - single-document, multi-hop, and unanswerable. Multi-hop counts as a hit only when every answer document makes it into the context; unanswerable items are scored on abstention, not recall.
  4. Justify the size: 20 items separate 'broken' from 'usable' and are enough for a smoke gate; 100 to 200 are needed before a two-point delta means anything. Then grow the set - every production failure becomes a new item.
  5. Mention cost and decay: roughly two hours for 20 items, and answer labels must be rechecked whenever the corpus changes, or the set rots and you misread the drop as a system regression.
  6. Expected follow-up: how do you avoid overfitting to the eval set? Keep a held-out slice that never informs tuning, and refresh it from real production questions.

  答题要点

  • 从语料反向出题，出题时答案文档就已确定，标注成本最低。
  • 每题标类型：单文档、多跳、无答案，三类缺一不可。
  • 多跳要求全部答案文档命中；无答案不算召回率，考的是拒答。
  • 20 题够冒烟，100 到 200 题才能判断小幅变化；线上故障持续补题。
  • 留一份不参与调优的保留集，防止对评估集过拟合。

  Key points

  • Author corpus-first so the answer document is known at authoring time.
  • Label every item with a type: single-document, multi-hop, unanswerable.
  • Multi-hop requires all answer documents; unanswerable items score abstention, not recall.
  • 20 items for a smoke gate, 100 to 200 to trust small deltas, and keep growing it from production failures.
  • Hold out a slice that never informs tuning to avoid overfitting the set.
• 召回率、平均倒数排名、归一化折损累计增益，这三个检索指标分别在什么故障下会先掉下来？只盯一个会漏掉什么？Recall, mean reciprocal rank, and normalized discounted cumulative gain - which failure mode does each one catch first, and what do you miss by watching only one?

  国内高频海外高频进阶#retrieval-metrics#evaluation#ranking

  分析过程 · 先想清楚再作答

  1. 这题考的是「知不知道指标之间的盲区」，不是背定义。能把三者按「有没有 / 靠不靠前 / 整体好不好」分层的，基本就答对了一半。
  2. 推导链是这样的：召回率是布尔的——答案文档在不在最终上下文里。它对「压根没捞到」最敏感，但答案从第 1 名掉到第 8 名它一动不动，只要还在预算内。
  3. 倒数排名只看第一条相关结果的名次，所以「答案还在但被挤到后面」它立刻掉。反过来它有个盲区：前十条里有一条命中还是五条命中，它给的分完全一样。
  4. 归一化折损累计增益把前 k 名里每一条相关结果都按名次折算再累加，所以它对「整体排序质量」敏感，是重排最直接的优化目标。它的盲区是不告诉你「有没有」——召回率为零时它也是零，看不出是没捞到还是排得差。
  5. 结论：三个一起看才能定位故障层。召回率掉说明检索或切块出了问题，要动召回策略；召回率不动而倒数排名掉，说明排序退化，该上重排；两者都稳而 nDCG 掉，说明前几名里混进了更多噪声。
  6. 可预期的追问是「指标顶格了怎么办」。真实答案是把题目做难：指标撞天花板说明评估集失去区分度，这时候继续优化系统是在瞎调。

  How to reason about it · think before answering

  1. This tests whether you know each metric's blind spot, not whether you can recite definitions. Layer them as 'did it show up / how high / how good overall' and you are halfway there.
  2. Recall is boolean: is the answer document in the final context. It catches 'never retrieved', but it does not move when the answer slips from rank 1 to rank 8, as long as it still fits the budget.
  3. MRR looks only at the rank of the first relevant hit, so ranking degradation shows up immediately. Its blind spot: one relevant item in the top ten scores exactly the same as five.
  4. nDCG discounts every relevant hit in the top k by its position, so it tracks overall ranking quality and is the direct optimization target for reranking. Its blind spot is existence - it is zero both when nothing was retrieved and when ranking is terrible.
  5. Conclusion: together they localize the failure. Recall drops means retrieval or chunking; recall flat but MRR down means ranking degraded, reach for a reranker; both stable but nDCG down means more noise crept into the top results.
  6. Expected follow-up: what if a metric saturates? Make the questions harder - a saturated metric means the eval set lost its discriminative power, and further tuning is blind.

  答题要点

  • 召回率管「有没有进上下文」，对完全没捞到最敏感，对名次变化不敏感。
  • 平均倒数排名管「第一条排第几」，对排序退化最敏感，但分不清命中一条还是五条。
  • 归一化折损累计增益管「前 k 名整体质量」，是重排的直接优化目标，但看不出有没有。
  • 三者组合才能定位故障在召回层、排序层还是噪声层。
  • 命中口径要说清：按 token 预算装上下文，不是按固定条数取前 k。

  Key points

  • Recall answers 'did it make it into the context', sensitive to total misses, blind to rank shifts.
  • MRR answers 'how high is the first hit', sensitive to ranking degradation, blind to how many hits there are.
  • nDCG answers 'how good is the top k overall', the direct target for reranking, blind to existence.
  • Only the combination localizes the failure to retrieval, ranking, or noise.
  • State the hit criterion: context is packed against a token budget, not a fixed top-k.

D9 混合检索与重排：两路召回、倒数排名融合，再用交叉编码器把前几名重新排一遍

• 混合检索为什么普遍用倒数排名融合，而不是把两路分数归一化之后加权相加？加权那条路在什么情况下会失控？Why do hybrid retrieval systems usually use reciprocal rank fusion instead of normalizing both scores and adding them with weights? When does the weighted approach break down?

  国内高频海外高频进阶#hybrid-search#rank-fusion

  分析过程 · 先想清楚再作答

  1. 这题的题眼在「分数」两个字。只答「RRF 更简单」是背概念，面试官想听的是你知道分数为什么不可比。
  2. 先给量纲差异：BM25 是一堆对数项累加，没有上界，同一套索引里不同查询的第一名可以从 5 分到 50 分；余弦被钉死在负一到正一。两个读数相加没有意义。
  3. 再点出归一化的静默失败：除以本路最高分之后，分母随查询浮动。一个语料里根本没有答案的问题，向量那一路最高分只有 0.09，归一化之后照样是满分 1.0 带权重进融合——你以为在比相关性，其实在比「本路矮子里有多高」。
  4. 然后是权重的维护成本：1 比 0.6 这个配比要靠跑评估调出来，两路是二维搜索，加上多路查询就是四维五维，而且换一个 embedding 模型全部作废。RRF 只有一个 k，而且 60 这个默认值几乎不用动。
  5. 结论：名次是两路唯一可比的东西。RRF 主动扔掉分数，是为了不被不可比的量误导。
  6. 可预期的追问：那 k 是干什么的？答 k 是压平器——k 越大，头几名之间的差距越小，于是「两路都排进前列」比「一路排第一」更有分量，这正是混合检索想要的交叉验证效果。再追问同分怎么办，答必须按文档 id 兜底排序，否则跨次运行名次会飘、评估数字跟着抖。

  How to reason about it · think before answering

  1. The hinge word is `scores`. Answering `RRF is simpler` is reciting a concept; the interviewer wants to hear that you know why the two scores are not comparable in the first place.
  2. Start with scale: BM25 is an unbounded sum of log terms, and on one index the top hit can range from 5 to 50 depending on the query; cosine is pinned between -1 and 1. Adding those two readings is meaningless.
  3. Then name the silent failure of normalization: dividing by the per-route maximum makes the denominator float with the query. For a question with no answer in the corpus, the vector route's best hit may score 0.09 and still normalize to a perfect 1.0, entering the fusion at full weight. You think you are comparing relevance; you are comparing `tallest among the short`.
  4. Then the maintenance cost of weights: a 1-to-0.6 ratio has to be tuned against an eval set, tuning two routes is a 2-D search, adding multi-query retrieval makes it 4-D or 5-D, and swapping the embedding model invalidates all of it. RRF has a single k, and the default of 60 rarely needs touching.
  5. Conclusion: rank is the only thing the two routes share. RRF throws the scores away on purpose so that an incomparable quantity cannot mislead it.
  6. Expected follow-up: what does k do? It flattens — the larger k is, the smaller the gap between the top few ranks, so `ranked well by both routes` outweighs `ranked first by one route`, which is exactly the cross-validation effect hybrid retrieval is after. A second follow-up on ties: you must fall back to sorting by document id, or ranks drift between runs and every eval number wobbles with them.

  答题要点

  • BM25 无上界、余弦有界，两个量纲不可比，直接相加没有意义。
  • 按本路最高分归一化的分母随查询浮动，无答案的查询里最不相干的结果也能拿到满分。
  • 权重要跑评估调，路数一多就是高维搜索，换模型还得重来；RRF 只有一个常数 k。
  • RRF 只吃每一路的有序 id 列表，名次是两路唯一可比的东西。
  • k 越大越奖励「两路都排进前列」；同分必须按 id 兜底排序才可复现。

  Key points

  • BM25 is unbounded, cosine is bounded; the two scales are not comparable, so adding them is meaningless.
  • Per-route max normalization has a denominator that floats with the query, so the least relevant hit of an unanswerable query still normalizes to 1.0.
  • Weights must be tuned against an eval set, the search is high-dimensional once you add routes, and swapping models invalidates it; RRF has a single constant k.
  • RRF consumes only the ordered id list from each route, because rank is the one thing the routes share.
  • Larger k rewards `ranked well by both routes`; ties must fall back to document id so results are reproducible.

D10 查询侧优化：改写、假设文档嵌入、多路查询、后退提问与意图路由

• 假设文档嵌入（HyDE）为什么有效？它在什么情况下会把检索带偏？Why does HyDE (hypothetical document embeddings) work, and when does it steer retrieval in the wrong direction?

  国内高频海外高频进阶#hyde#query-transformation#retrieval-quality

  分析过程 · 先想清楚再作答

  1. 这题的题眼在后半句。前半句网上到处都能抄到，能不能说清「什么时候不该用」才是区分度所在——只答前半句的人，多半没在真实语料上跑过。
  2. 先给机制：向量检索比的是语义相似度，而用户的疑问句和文档里的制度条文在文体、句式、用词上都不同类。HyDE 先让模型编一段「长得像目标文档」的假文本，用它的向量去找邻居，等于把查询搬进了文档所在的那个语域。
  3. 紧接着点破一个常见误解：这段假文本的**事实对不对根本不重要**，因为它不给用户看，只贡献一个向量方向。理解到这一层，才算真懂它为什么不怕模型瞎编。
  4. 带偏有两种典型情况。一是模型编得太具体，给出语料里根本不存在的字段名或流程名，向量朝着一个不存在的方向去了；二是语料里压根没有答案，本该拒答的问题被编出来的假文档匹配到几个「看起来挺像」的邻居，拒答率掉下去、瞎编率涨上来。
  5. 说完风险要给对策，这一步最见工程经验：门槛卡在**每一路检索器的原始分**上而不是融合分上（融合分是相对的，最不相干的一批也能拿最高分）；以及把假设文档当成**第二个检索式与原问题融合**，而不是直接替换原问题——替换在模型编歪时会把原问题的信号一起丢掉。
  6. 可预期的追问是「它多花多少钱」。答：假设文档要写上百字，输出 token 是查询改写的十几倍，是查询侧四种手法里最贵的一次调用，而且检索次数翻倍。所以它通常不该默认打开，应该进 A/B 队列。

  How to reason about it · think before answering

  1. The tell is in the second half. Anyone can recite why HyDE works; only someone who has run it on real data can say when it hurts.
  2. Give the mechanism first: dense retrieval compares semantic similarity, but a user's question and a policy paragraph differ in register, syntax and vocabulary. HyDE has the model draft a fake passage that looks like the target document, then retrieves with that vector — effectively moving the query into the documents' register.
  3. Then kill the common misreading: the factual accuracy of the draft does not matter, because it is never shown to the user. It only contributes a direction in embedding space.
  4. Two failure modes. The model invents an over-specific field or process name that does not exist in the corpus, and the vector chases something imaginary. Or the corpus genuinely has no answer, and the fabricated passage finds plausible-looking neighbours anyway — abstention rate drops and hallucination rate climbs.
  5. Pair the risk with a mitigation: gate admission on each retriever's raw score, never on the fused score (fused scores are relative, so even the worst batch tops out at 1.0); and treat the hypothetical document as a second query fused with the original rather than a replacement, so a bad draft can only dilute the signal, not erase it.
  6. Expect the follow-up on cost. The draft runs to a hundred-plus output tokens, an order of magnitude more than a rewrite, and it doubles retrieval calls. That is why it belongs in an A/B queue, not in the default config.

  答题要点

  • 有效的原因是语域对齐：疑问句和制度条文本来不在一个语义邻域，假设文档把查询搬到了文档那一侧。
  • 假文本的事实对错不重要，它只贡献一个向量方向，不展示给用户。
  • 带偏的两种情况：编得太具体，追一个语料里不存在的方向；本该拒答的问题被假文档匹配上，拒答率下降。
  • 两条护栏：门槛卡原始分不卡融合分；把假设文档当第二个检索式融合，而不是替换原问题。
  • 成本上它是查询侧最贵的一项（长输出加检索次数翻倍），默认关闭、按场景 A/B。

  Key points

  • It works by register alignment: a question and a policy paragraph sit in different neighbourhoods, and the fake passage moves the query into the document's.
  • The draft's factual accuracy is irrelevant — it only supplies a direction and is never shown to the user.
  • It misfires when the model invents over-specific details, or when the corpus has no answer and the fabrication finds plausible neighbours anyway.
  • Two guardrails: gate on raw per-route scores, not fused ones; fuse the hypothetical document with the original query instead of replacing it.
  • It is the most expensive query-side technique (long output plus doubled retrievals), so keep it off by default and A/B it.
• 意图路由判错了会怎样？你会怎么设计兜底？What happens when intent routing misclassifies, and how would you design the fallback?

  国内高频海外高频进阶#intent-routing#fallback#observability

  分析过程 · 先想清楚再作答

  1. 这题在考「有没有想过错误的方向」。路由是分类器，分类器一定会错；只答「多加训练数据提高准确率」的，等于没回答兜底怎么设计。
  2. 先把错误按方向拆开，这一步是整题的骨架：三条路（直接回答、单跳检索、多跳检索）两两误判，代价完全不对称。把该检索的判成直接回答，模型手里一点材料都没有，只能编，这是最贵的一种错；把闲聊判成单跳，只是白花一次检索；把多跳判成单跳，只是少查一轮、答得不全。
  3. 结论顺势就出来了：**兜底方向要偏向「多花一点钱」，判不出来一律退回单跳检索。** 单跳是三条路里错得最轻的一条，而且它的错误是可恢复的——材料不全模型还能说「资料里只查到一半」，材料为空它就只能编。
  4. 再补一层运行时兜底，比事前分类更管用：分类成直接回答之后，如果模型的回答里出现了具体数字、金额、日期这类需要出处的内容，就回退去检索一次再答；分类成单跳之后，如果检索侧一条都没过门槛，就升级走多跳或直接拒答。**用后一步的观测结果纠正前一步的判断**，这是路由系统最实用的一条设计。
  5. 还要提一句可观测性：路由的每一次判定都要落日志，带上原始问题、判定结果、后续是否发生了兜底升级。没有这份日志，你既不知道路由准不准，也没法攒出下一版的训练集。
  6. 可预期的追问是「什么时候干脆别做路由」。答：流量里闲聊占比很低、且多跳问题很少时，路由省下的钱还不够付分类调用的钱，这时候直接全部走单跳更划算——我们在 30 篇语料的实验里就看到，路由真正的收益并不在省检索，而在于认出多跳之后给它更高的上下文预算。

  How to reason about it · think before answering

  1. This tests whether you have thought about the direction of the error. A router is a classifier and classifiers misfire; "add more training data" is not a fallback design.
  2. Break the errors down by direction — that is the backbone of the answer. Across three routes (direct answer, single-hop, multi-hop) the six confusions carry wildly asymmetric costs. Routing a retrieval-worthy question to a direct answer leaves the model with no material at all, so it fabricates: the most expensive error. Routing chit-chat to single-hop merely wastes one retrieval. Routing multi-hop to single-hop just yields an incomplete answer.
  3. The conclusion follows: bias the fallback toward spending a little more, and default to single-hop retrieval whenever the classifier is unsure. Single-hop is the cheapest error to make, and it is recoverable — with partial material the model can still say it only found half the answer; with no material it can only invent one.
  4. Add a runtime fallback, which beats better up-front classification: after a direct-answer routing, if the draft reply contains figures, amounts or dates that need a source, fall back to retrieval and answer again; after a single-hop routing, if no candidate clears the admission gate, escalate to multi-hop or abstain. Correcting the earlier decision with the later observation is the single most useful pattern in routing systems.
  5. Mention observability: log every routing decision with the raw question, the label, and whether a fallback fired. Without that log you know neither how accurate the router is nor what to train the next version on.
  6. Expect "when should you skip routing entirely?" When chit-chat is a small share of traffic and multi-hop questions are rare, the classification call costs more than it saves. In our 30-document lab the real gain from routing was not saved retrievals but the ability to give recognised multi-hop questions a larger context budget.

  答题要点

  • 三条路的误判代价不对称：把该检索的判成直接回答最贵（模型没材料只能编），把闲聊判成单跳只是白花一次检索。
  • 兜底方向偏向多花钱：判不出来一律退回单跳检索，它是错得最轻且可恢复的一条路。
  • 加运行时兜底：直接回答里出现需要出处的数字就补一次检索；单跳检索一条都没过门槛就升级或拒答。
  • 每一次路由判定都落日志（原始问题、判定结果、是否触发兜底），既用于监控也用于攒下一版训练集。
  • 闲聊与多跳占比都很低时，路由省的钱付不起分类调用，直接全走单跳更划算。

  Key points

  • The three routes have asymmetric error costs: sending a retrieval-worthy question to a direct answer is the worst, while routing chit-chat to single-hop only wastes one retrieval.
  • Bias the fallback toward spending more: default to single-hop whenever the classifier is unsure, since that error is the mildest and is recoverable.
  • Add runtime fallbacks: re-retrieve if a direct answer contains figures that need a source; escalate or abstain if no single-hop candidate clears the gate.
  • Log every routing decision — raw question, label, whether a fallback fired — for both monitoring and the next training set.
  • When chit-chat and multi-hop are both rare, the classification call costs more than it saves; route everything to single-hop instead.

D11 高级索引：父子文档、摘要索引、上下文检索，以及树状聚合与图检索的取舍

• 父子索引和上下文检索都在补『块被切碎』这个问题，它们的差别到底在哪？Parent-child indexing and contextual retrieval both patch the same problem — chunks losing their context. What actually distinguishes them?

  国内高频海外高频进阶#indexing#contextual-retrieval#chunking

  分析过程 · 先想清楚再作答

  1. 这题的题眼是『补的是哪一半』。答成『一个是切块技巧、一个是加提示词』就是在描述实现，面试官想听的是它们各自作用在检索管道的哪一段。
  2. 拆的办法是把管道分成两段问：检索时看到什么、生成时看到什么。父子索引改的是**生成侧**——检索单位还是小块，只是命中之后把上下文单位换成大块；上下文检索改的是**检索侧**——块头拼进去是为了让这一块能被检索到，模型生成时并不需要它。
  3. 结论：父子索引解决『找到了但看不全』，上下文检索解决『看得全但找不到』。前者不改变谁被检索到，后者不改变模型看到多少。它们正交，可以叠加。
  4. 这个差别还决定了它们各自要用什么指标去量：上下文检索动的是名次，用召回率和 nDCG 量得到；父子索引动的是『材料够不够答』，召回率这种二值指标量不出来。我们那份 20 题评估集单文档档已经 100% 饱和，父子索引在表里跟基线持平——那不是它没用，是尺子量不了它。
  5. 顺着这条差异能推出一个立刻能用的优化：既然块头只服务检索，就不该进上下文。它进了上下文就是在每一次查询里白占预算，而且这笔钱是长期的。我们的实验里把这个开关一改，五列指标一个不变，600 token 的预算里多装进了 36 个 token。
  6. 代价也不同：父子索引的代价是索引条目变多、每次装进上下文的东西变大；上下文检索的代价是一次性要给每块调一次模型，加上索引 token 永久变大。前者是空间，后者是时间加空间。
  7. 可预期的追问是『那我全都上』。答案是先看失败案例：日志里是『材料不完整』多，还是『压根没检索到』多。没有对应的失败模式就不该上，这两个手法都不是免费的。

  How to reason about it · think before answering

  1. The hinge is which half of the pipeline each one fixes. Answering 'one is a chunking trick, the other adds a prompt' just describes implementations; the interviewer wants to know where each acts.
  2. Split the pipeline in two and ask separately: what does the retriever see, and what does the generator see. Parent-child changes the generation side — retrieval still runs on small chunks, but a hit is swapped for its parent. Contextual retrieval changes the retrieval side — the header exists so the chunk can be found at all, and the generator does not need it.
  3. Conclusion: parent-child fixes 'found it but can't read it'; contextual retrieval fixes 'readable but never found'. Neither changes what the other changes, so they compose.
  4. That difference also dictates which metric can see each one. Contextual retrieval moves rank, so recall and nDCG catch it. Parent-child moves 'is the evidence sufficient to answer', which a binary recall metric cannot see. Our 20-question set is already saturated at 100% on single-document questions, so parent-child comes out level with the baseline — that is the ruler failing, not the technique.
  5. That difference yields a free optimization: since the header only serves retrieval, keep it out of the context window. Leaving it in pays rent on every single query. Flipping that one switch in our lab freed 36 tokens inside a 600-token budget with every metric unchanged.
  6. The costs differ too. Parent-child costs index entries and a bigger context unit. Contextual retrieval costs one model call per chunk up front plus a permanently larger index. One is space; the other is time and space.
  7. Expect the follow-up 'why not both'. Look at the failure logs first: are you mostly seeing incomplete evidence, or nothing retrieved at all? Without the matching failure mode, neither is worth its price.

  答题要点

  • 父子索引作用在生成侧：检索单位是小块，上下文单位换成父块，解决『找到了但看不全』。
  • 上下文检索作用在检索侧：块头让块能被检索到，解决『看得全但找不到』。
  • 两者正交可叠加；块头只该进索引不该进上下文，否则每次查询都在为它付钱。
  • 父子索引的代价是索引条目与上下文单位变大；上下文检索的代价是一次性建索引调用加永久变大的索引。
  • 选哪个看失败案例：材料不完整选前者，压根没检索到选后者。

  Key points

  • Parent-child acts on the generation side: retrieve small, swap in the parent for context. It fixes 'found but unreadable'.
  • Contextual retrieval acts on the retrieval side: the header makes the chunk findable. It fixes 'readable but never found'.
  • They are orthogonal and compose; keep the header in the index only, never in the context window.
  • Parent-child costs more index entries and a larger context unit; contextual retrieval costs one call per chunk plus a permanently larger index.
  • Pick based on the observed failure: incomplete evidence points to the former, zero retrieval to the latter.
• 同一份语料建了三套索引，检索时你怎么决定走哪一套？You have built three different indexes over the same corpus. How do you decide which one a query goes to?

  国内高频海外高频进阶#index-routing#evaluation#architecture

  分析过程 · 先想清楚再作答

  1. 这题是送分还是丢分，取决于你有没有先反问一句『真的需要三套吗』。上来就答路由策略的人，默认了一个没被验证的前提。
  2. 第一步是承认多数情况下答案是『都不走，走默认那套』。我们在 30 篇语料上把五种索引结构各测一遍，**召回率全部停在 93.8%，没有一种跑赢基线**；唯一动了的是 nDCG@10（块头把它从 0.6438 抬到 0.7218），而两段式的摘要索引还掉到了 87.5%。每种结构补的都是一个特定短板，你没有那个短板时它只带来成本。
  3. 第二步才是路由，而判据不是『哪套准』——那是离线评估该回答的问题，不是运行时能知道的。运行时能拿到的只有**问题的形状**：细节型（答案落在某一段）、概括型（要全库的一个概括）、多跳型（要跨实体串联）。按形状分流，正好对应块级索引、树状聚合索引、图索引。
  4. 实现上就是一个轻量意图分类器，跟前一天的意图路由是同一套东西，不必再造一个。分类结果作为元数据带进请求，方便事后拿评估集回看分错了多少。
  5. 兜底策略要说清楚：分类错了**回落到默认那一套**，不要并行全查一遍再融合。并行看着稳，实际上把延迟和成本按索引套数翻倍，而多出来的那两路大概率一条都进不了上下文预算。
  6. 可预期的追问是『怎么知道分类器分对了』。答案是把路由决策记进日志，定期拿标准答案集回放：对每个问题分别走三套索引，看分类器选的那套是不是指标最好的那套。这是一个能持续跑的离线作业，不需要人工标注。

  How to reason about it · think before answering

  1. Whether this is an easy point or a lost one depends on whether you first ask 'do we actually need three?'. Jumping straight to routing accepts an unverified premise.
  2. Step one is admitting the answer is usually 'none of them — use the default'. Across 30 documents we measured five index structures and every one landed at 93.8% recall, none beating the baseline. The only metric that moved was nDCG@10, which headers lifted from 0.6438 to 0.7218, while the two-stage summary index fell to 87.5%. Each structure patches one specific weakness; without that weakness it is pure overhead.
  3. Step two is routing, and the criterion is not 'which index is more accurate' — that is an offline evaluation question, not something you know at request time. What you do have at request time is the shape of the question: detail-seeking, summarizing, or entity-chaining. Those map onto the chunk index, the tree-summary index and the graph index.
  4. Implementation is a lightweight intent classifier — the same one from the previous day's intent routing, no need to invent another. Carry the decision as request metadata so you can replay it later.
  5. Spell out the fallback: on a misclassification, fall back to the default index rather than fanning out across all three and fusing. Fan-out looks safe but multiplies latency and cost by the number of indexes, and the extra routes usually never make it into the context budget anyway.
  6. Expect 'how do you know the classifier is right'. Log every routing decision and replay the golden set periodically: run each question through all three indexes and check whether the classifier picked the best-scoring one. It is a standing offline job that needs no human labelling.

  答题要点

  • 先反问是否真需要三套：实测五种索引结构召回率全部持平在 93.8%，没有对应短板就是纯成本。
  • 运行时的判据是问题的形状——细节型、概括型、多跳型，分别对应块级、树状摘要、图索引。
  • 复用前一天的意图路由做分类，把路由决策记进请求元数据。
  • 分类错了回落到默认索引，不要并行全查再融合——延迟和成本按套数翻倍。
  • 用标准答案集定期回放，检验分类器选的那套是不是指标最好的那套。

  Key points

  • First challenge the premise: all five index structures landed at the same 93.8% recall in our measurement, so an index without a matching weakness is pure cost.
  • At request time the usable signal is question shape — detail, summary, or entity-chaining — mapping to chunk, tree-summary and graph indexes.
  • Reuse the previous day's intent router for classification and record the routing decision as request metadata.
  • Fall back to the default index on misclassification instead of fanning out and fusing, which multiplies latency and cost.
  • Replay the golden set periodically to check whether the classifier picks the best-scoring index.

D12 Agentic RAG：把检索做成工具，让模型自己决定查不查、查几次、要不要推翻重来

• 自反思式检索会反复改写查询重试。你怎么保证它一定会停下来，而不是在同一个查询上原地打转？Self-reflective retrieval rewrites the query and retries. How do you guarantee it terminates instead of spinning on the same query forever?

  国内高频海外高频进阶#agentic-rag#self-reflection#reliability

  分析过程 · 先想清楚再作答

  1. 这题在考「有没有真让循环跑过」。只答「设一个最大轮数」的能拿一半分，因为最大轮数只拦住了一类失控，剩下两类照样漏出去。
  2. 怎么拆：把失控分成三种形态，每种配一道闸。一是「每轮都在推进但永远推进不完」，用最大轮数拦；二是「每轮都不超标但累计爆掉」，用累计 token 预算拦——四轮各读 600 token 没有一轮超标，可送进模型的材料已经是单轮的四倍；三是「原地打转」，用重复查询检测拦。
  3. 重复查询检测有两个实现细节，答出来就说明真写过：一是要放在检索之前，否则要白花一次调用才发现自己在转圈；二是判重要对查询做归一化，只看词的集合，否则「主备切换 审批」和「审批 主备切换」会被当成两个不同的查询，圈照转不误。
  4. 还要说清停下来之后怎么办：停止原因必须分类记录，「查够了」「主动认输」「撞到轮数」「撞到预算」「原地打转」是五种不同的结局。把它们混成一个「循环结束」，你就永远看不见系统在多大比例的问题上其实是放弃了。
  5. 一个容易被忽略的点：闸门装了不等于验过。默认预算如果比实际用量高一大截，跑多少遍都踩不响它，等于没装。每一道闸都要构造一个用例把它踩响，这是验收的一部分。
  6. 可预期的追问是「模型自己说不够，但其实已经够了怎么办」。答案是自评要给结构化输出（覆盖了哪些要素、缺哪些），缺失项为空却仍判不够时按「够了」处理——让判断可审计，而不是信一个布尔值。

  How to reason about it · think before answering

  1. This checks whether you have actually run such a loop. 'Set a max iteration count' is half an answer: it stops one failure mode and lets two others through.
  2. Split runaway behaviour into three shapes and give each its own brake. Progress that never completes is capped by max rounds. Per-round budgets that pass individually but blow up in aggregate need a cumulative token budget - four rounds of 600 tokens each never trips a per-round check yet quadruples what reaches the model. Spinning in place needs duplicate-query detection.
  3. Two implementation details prove you have written it: the duplicate check belongs before the retrieval call, otherwise you pay for a call to learn you are looping; and queries must be normalized to a set of terms, or 'failover approval' and 'approval failover' count as two distinct queries and the loop keeps turning.
  4. Say what happens after it stops: stop reasons must be recorded as distinct categories - satisfied, gave up, hit round cap, hit token budget, duplicate query. Collapsing them into 'loop finished' hides how often the system simply surrendered.
  5. An easy miss: installing a brake is not testing it. If the default token budget sits far above real usage it never fires, which is the same as not having one. Every brake needs a case that trips it.
  6. Expected follow-up: what if the model says 'not enough' when it actually is? Make the assessment structured - which elements are covered, which are missing - and treat an empty missing list as sufficient, so the decision is auditable rather than a bare boolean.

  答题要点

  • 三道闸缺一不可：最大轮数、累计 token 预算、重复查询检测。
  • 累计预算拦的是「每轮都不超但加起来爆掉」，轮数闸看不见这件事。
  • 重复查询检测要放在检索之前，且查询要归一化成词的集合再判重。
  • 停止原因分类记录：查够了、主动认输、撞轮数、撞预算、原地打转是五种结局。
  • 每一道闸都要构造用例踩响，装了没验过等于没装。
  • 自评输出结构化的覆盖与缺失项，让「不够」这个判断可审计。

  Key points

  • Three brakes, none optional: max rounds, cumulative token budget, duplicate-query detection.
  • The cumulative budget catches rounds that each pass but blow up together - the round cap cannot see that.
  • Check for duplicates before retrieving, and normalize the query to a term set before comparing.
  • Record stop reasons as distinct categories rather than one 'finished' bucket.
  • Every brake needs a case that actually trips it; an untested brake is no brake.
  • Have the assessor emit covered and missing elements so 'not enough' is auditable.
• 什么情况下你会拒绝把一个 RAG 系统做成 Agentic 的？拿什么数据说服你的团队？When would you refuse to make a RAG system agentic, and what data would you use to convince your team?

  国内高频海外高频进阶#agentic-rag#cost#engineering-judgement

  分析过程 · 先想清楚再作答

  1. 这题在考工程判断力，也在考你会不会算账。凡是答「Agentic 更先进所以要上」的，直接出局；面试官想听的是你能主动说出它的代价，并且用数字划出适用边界。
  2. 怎么拆：先承认收益来自哪一类问题，再看这类问题在你的流量里占多大比例。Agentic 的收益几乎全部集中在多跳和检索失败重试上，单文档可答的问题一次检索就够了，多查一轮纯属浪费。
  3. 所以判据不是感觉，是评估集：跑一遍，看 multi 那一档占多少题、涨了多少个点，再对照总调用次数涨了多少倍。在一份 20 题的集合上，我们量到的是多跳召回从 75% 涨到 100%，可答题整体只从 93.8% 涨到 100%，代价是平均检索调用从 1 次涨到 1.75 次、外加同样次数的自评调用——为 100% 的问题付钱，只有 5% 的问题拿到好处。
  4. 三类明确不上：延迟敏感（每多一轮就是一次检索加一次模型往返，首字延迟拉长一到两倍）；问题模式固定（九成是单文档可答，收益接近零）；成本吃紧（真实模型不像离线替身那样老实，成本方差比均值更难受，按均值做的容量规划会在长尾上被打穿）。
  5. 给出替代方案才算完整：分流。先用一次便宜的判断看这一问像不像多跳，像才进循环，不像走固定流程。九成走一次检索、一成走循环，账完全不一样。这也说明循环是一种能力，不是默认值。
  6. 可预期的追问是「那你怎么知道哪些问题像多跳」。答案是从评估集和线上日志里找模式（问句里同时问了两个事实、问的是某个角色背后的人），先用规则跑，跑不动再上小模型分类——顺序不要反。

  How to reason about it · think before answering

  1. This tests engineering judgement and whether you can do arithmetic. Anyone who says 'agentic is more advanced so we should ship it' is out. The interviewer wants you to name the cost and draw the boundary with numbers.
  2. Decompose it: identify which question types actually benefit, then check how much of your traffic they represent. Agentic gains concentrate in multi-hop questions and retrieval retries; single-document questions are answered by one lookup and every extra round is waste.
  3. So the criterion is the evaluation set, not intuition. On a 20-item set we measured multi-hop recall going from 75% to 100% while overall answerable recall moved only from 93.8% to 100%, at the cost of average retrieval calls going from 1 to 1.75 plus the same number of assessment calls - you pay for 100% of traffic so that 5% of it improves.
  4. Three clear refusals: latency-sensitive surfaces, where each round adds a retrieval plus a model round trip and roughly doubles time to first token; fixed question patterns, where nine in ten questions are single-document and the gain is near zero; and tight cost budgets, where a real model is less disciplined than an offline stand-in and the variance, not the mean, is what breaks your capacity plan.
  5. Finish with the alternative: route. Use one cheap check to decide whether a question looks multi-hop, and only then enter the loop. Nine tenths take a single retrieval, one tenth loops, and the economics change completely. Looping is a capability, not a default.
  6. Expected follow-up: how do you know which questions look multi-hop? Mine the eval set and production logs for patterns - two facts requested in one sentence, or a question about the person behind a role - start with rules, and reach for a small classifier only when rules stop working.

  答题要点

  • 收益集中在多跳与检索失败重试，单文档可答的问题上收益接近零。
  • 用评估集算账：multi 档涨了多少点，对照总调用次数涨了多少倍。
  • 实测过的一组数字：多跳召回 75% 到 100%，整体 93.8% 到 100%，检索调用 1 次到 1.75 次外加等量自评调用。
  • 三类不上：延迟敏感、问题模式固定、成本吃紧（方差比均值更难受）。
  • 替代方案是分流：便宜的判断先过滤，像多跳才进循环。
  • 循环是一种能力，不是默认值。

  Key points

  • Gains concentrate in multi-hop and retry cases; single-document questions gain almost nothing.
  • Settle it with the evaluation set: multi-hop delta against the multiplier on total calls.
  • One measured set: multi-hop recall 75% to 100%, overall 93.8% to 100%, retrieval calls 1 to 1.75 plus the same number of assessment calls.
  • Refuse when latency-sensitive, when question patterns are fixed, or when cost is tight - variance hurts more than the mean.
  • Route instead: a cheap check up front, and only multi-hop-looking questions enter the loop.
  • Looping is a capability, not a default.

D13 上生产：增量同步与去重、按权限过滤、缓存分层、链路追踪与成本延迟账

• 文档更新之后，你怎么做到只重算受影响的块？被删掉的文档又怎么保证一定从索引里消失？After a document changes, how do you recompute only the affected chunks? And how do you guarantee a deleted document really disappears from the index?

  国内高频海外高频进阶#incremental-sync#content-hash#index-maintenance

  分析过程 · 先想清楚再作答

  1. 这题有两半，区分度全在后半。前半几乎人人答得出「算个哈希比一比」，能不能拿到分取决于你有没有主动讲删除——那是同一套机制里唯一不对称的一种变更。
  2. 先给增量的骨架：拿来源的全集和索引的全集做三向对账。来源有、索引没有是新增；两边都有但内容指纹不同是修改；索引有、来源没有是删除。修改的处理是整篇替换，先删旧块再写新块，不能只追加——不然改短了的文档会在索引里留下一截尾巴。
  3. 接着讲指纹本身，这是给分点：sha256 取前若干位，但**算之前必须先做换行归一化再去首尾空白**。同一份文件从 Windows 传一次、从 Mac 传一次，字节不同内容相同，不归一化就每次都判成变了，等于天天在做全量重建。这个 bug 不报错，只体现在账单上。
  4. 然后是删除这一半的关键判断：**删除不是一个事件，是一个缺席**。文件变动类的通知只告诉你哪些东西变了，永远不会有人发一条「我不存在了」。所以删除检测必须反着来——遍历索引，找出来源里已经没有的 id。只监听变更事件的同步器永远等不到这条消息。
  5. 落到存储上：文档、块、向量三张表用外键级联删除，删文档只写一条语句，剩下的交给数据库。手写三条删除的版本迟早会漏掉一条，而漏掉的那条就是索引里的幽灵。收尾时报一个可验证的指标：块数与向量数必须相等，不等就说明有孤儿。
  6. 可预期的追问：来源系统本身就不可靠、拉不全怎么办？那就把「本次拉取是否完整」当成删除检测的前置条件——拉取不完整时只做新增和修改，不做删除，否则一次拉取失败会把半个索引清空。另外给删除加软删标记和保留期，误删还能回滚。

  How to reason about it · think before answering

  1. There are two halves here and the second one separates candidates. Almost everyone can say 'hash it and compare'; the score comes from bringing up deletion yourself, because it is the one asymmetric case in the whole mechanism.
  2. Give the skeleton first: a three-way reconciliation between the full set from the source and the full set in the index. In source but not indexed is an add; in both but with different content hashes is a modify; indexed but absent from the source is a delete. A modify must replace the document wholesale, deleting old chunks before writing new ones, otherwise a shortened document leaves a tail behind in the index.
  3. Then the fingerprint itself, which is where points are won: sha256 truncated, but normalize line endings and trim before hashing. The same file uploaded from Windows and from macOS differs byte-wise but not in content; skip normalization and every re-upload counts as a change, which is a full rebuild in disguise. It never raises an error, it only shows up on the bill.
  4. The key insight in the second half: a deletion is not an event, it is an absence. Change feeds tell you what changed; nobody ever sends 'I no longer exist'. So deletion detection has to run in the opposite direction — walk the index and find ids the source no longer has. A synchronizer that only listens to change events will wait forever.
  5. At the storage layer, cascade the foreign keys across documents, chunks and embeddings so deleting a document is a single statement and the database does the rest. Hand-written three-step deletes eventually miss one, and the one they miss is a ghost in the index. Close with a verifiable invariant: chunk count must equal embedding count, and a mismatch means orphans.
  6. Expected follow-up: what if the source system itself is unreliable and a pull comes back incomplete? Make pull completeness a precondition for deletion: on a partial pull, apply adds and modifies only, or one failed fetch wipes half your index. Also soft-delete with a retention window so a mistake is recoverable.

  答题要点

  • 三向对账：新增、修改、删除，缺一不可；修改是整篇替换，先删旧块再写新块。
  • 内容指纹算之前必须先做换行归一化再 trim，否则跨系统重传会被误判为修改，等于天天全量重建。
  • 删除是缺席不是事件，必须反过来遍历索引找出来源里已消失的 id，不能只监听变更通知。
  • 文档、块、向量用外键级联删除，删文档只写一条语句；用「块数等于向量数」当可验证的收尾指标。
  • 来源拉取不完整时只做新增与修改、跳过删除，并给删除加软删与保留期以便回滚。

  Key points

  • Three-way reconciliation covering adds, modifies and deletes; a modify replaces the whole document, old chunks first.
  • Normalize line endings and trim before hashing, or cross-platform re-uploads look like edits and you are doing a full rebuild every night.
  • Deletion is an absence, not an event: walk the index for ids the source no longer has instead of waiting on a change feed.
  • Cascade deletes from documents to chunks to embeddings so one statement suffices; assert chunk count equals embedding count to catch orphans.
  • On an incomplete pull, apply adds and modifies only, and soft-delete with a retention window so mistakes are reversible.
• RAG 系统里有哪些东西可以缓存？各自的失效条件是什么？What can be cached in a RAG system, and what are the invalidation conditions for each?

  国内高频海外高频进阶#caching#invalidation#cost-optimization

  分析过程 · 先想清楚再作答

  1. 这题看起来是送分题，实际是筛人题。答成「把问答结果缓存起来」只拿到三分之一，面试官等着听的是「分几层」和「各自什么时候失效」。
  2. 先给一条能迁移到别的题上的判断依据：**「什么时候必须失效」这个问题，等价于「key 里有没有把那样东西算进去」。** key 少放一样，那样东西变了缓存就不会失效。有了这条，三层的答案自己就长出来了。
  3. 然后逐层给：答案层缓存问题到最终答案，key 要有问题、权限范围、索引版本、模型与提示词版本；检索层缓存检索式到命中块列表，key 要有问题、权限范围、topK、索引版本、向量后端，但不需要模型；向量层缓存文本到向量，key 只有文本和向量后端。
  4. 重点讲向量层的反直觉之处：它是**内容寻址**的，文本没变、模型没变，向量就不会变，所以**不能把索引版本放进它的 key**。放进去的话一次同步就作废几万条向量，正好绕回全量重建——你加缓存想省的那笔钱又花回去了。这一层可以放很久甚至持久化。
  5. 给一个具体的失效手法：用**索引版本号**而不是精确删除。同步只要真的改动了索引就把版本号加一，旧 key 再也算不出来，自然没人读得到。精确删除要求你能列出「这次改动影响了哪些问题」，而那是列不出来的。
  6. 可预期的追问：能举一个「该失效却没失效」的真实例子吗？答：答案缓存的 key 只放了问题本身，文档里的上限从 200 MB 改成 500 MB、索引已经更新，再问同一个问题仍然返回 200 MB。它不报错，日志上是一次漂亮的缓存命中；同一个 key 还会让另一个部门的用户直接命中别人的答案。

  How to reason about it · think before answering

  1. This looks like a giveaway and is actually a filter. 'Cache the question and answer' earns a third of the credit; the interviewer is waiting for the layering and the per-layer invalidation rules.
  2. Lead with a transferable rule: 'when must this be invalidated' is the same question as 'is that thing part of the key'. Leave something out of the key and changes to it will never invalidate the entry. With that rule the three layers derive themselves.
  3. Then go layer by layer. The answer layer maps a question to a final answer; its key needs the question, the permission scope, the index version, and the model plus prompt version. The retrieval layer maps a query to a hit list; its key needs the question, scope, topK, index version and embedding backend, but not the generation model. The embedding layer maps text to a vector; its key is just the text and the backend.
  4. Emphasize the counterintuitive part of the embedding layer: it is content-addressed, so the index version must not be in its key. Put it there and a single sync invalidates tens of thousands of vectors, which is exactly the full rebuild you added caching to avoid. This is the one layer that can live a long time, even on disk.
  5. Offer a concrete invalidation mechanism: version numbers rather than targeted deletion. Bump an index version whenever a sync actually changes something and old keys simply stop being computed. Targeted deletion would require enumerating which questions a change affected, and that list cannot be produced.
  6. Expected follow-up: can you give a real 'should have expired but didn't' case? Yes: an answer cache keyed only on the question. A document's limit changes from 200 MB to 500 MB, the index is updated, and the same question still returns 200 MB. Nothing errors; the log shows a clean cache hit. The same key also serves one department's answer to a user from another.

  答题要点

  • 分三层：答案、检索、向量，三者的寿命差着数量级，不能当成一件事。
  • 判断依据是「什么时候必须失效」等价于「key 里有没有算进那样东西」，key 少一样就永远失效不了。
  • 答案层 key 要有问题、权限范围、索引版本、模型与提示词版本；检索层去掉模型、加上 topK 与向量后端。
  • 向量层是内容寻址的，key 只有文本与后端；把索引版本放进去会让每次同步都退化成全量重建。
  • 用索引版本号做失效比精确删除可靠，因为「这次改动影响了哪些问题」根本列不出来。

  Key points

  • Three layers — answer, retrieval, embedding — with lifetimes orders of magnitude apart; treating them as one thing is the mistake.
  • The rule is that 'when must it expire' equals 'is it in the key'; anything left out of the key can never invalidate the entry.
  • The answer key carries question, permission scope, index version, model and prompt version; the retrieval key drops the model and adds topK and the embedding backend.
  • The embedding layer is content-addressed and keyed only on text plus backend; adding an index version turns every sync back into a full rebuild.
  • Version-based invalidation beats targeted deletion because you cannot enumerate which questions a given change affected.

D14 综合项目与复盘：多租户企业知识库问答，一张 RAG 决策地图与面试专题

• RAG 系统上线后用户反馈「答得不准」，你的排查顺序是什么？Users report that your live RAG system 'answers inaccurately'. What is your triage order?

  国内高频海外高频进阶#debugging#failure-modes#observability

  分析过程 · 先想清楚再作答

  1. 这题几乎是必考题，而绝大多数人答成一堆并列的可能性：可能是切块问题、可能是提示词问题、可能是模型不行。并列不是排查，排查的意思是**有顺序、有判据、每一步能把可能性砍掉一半**。
  2. 先把「答得不准」这四个字拆开——它至少塞了四种病，而且修法互不通用：答非所问、只答得出片段、引用错位、更新不生效。所以第一个动作不是改配置，是**拿到具体的问题和回答，把它归到这四类里的一类**。
  3. 然后给顺序，而且要说清顺序的理由：**排查从右往左看、修复从左往右修**。从右往左是因为你最先看到的是生成结果；从左往右是因为上游的错会被下游放大——检索没捞到的东西，再好的提示词也救不回来。具体走法是：打印这一问的候选池和最终上下文，先看答案文档在不在候选池里。不在，是检索的债；在候选池但没过准入门槛，是门槛的债；过了门槛却没装进上下文预算，是块太大或预算太小；都进了而模型没用上，才轮到生成侧。
  4. 这里有一个容易写错的细节值得主动讲：**多跳题的诊断对象是缺的那几篇，不是「有没有捞到任意一篇」**。我们实验里有一道题要同时命中两篇，第一篇稳稳排第一、第二篇一次都没进候选池；用「任意一篇」去判会把它归成预算问题，然后你去调预算，调一整天也没用。这一条区分度很高，因为它只有真的按题排查过才想得到。
  5. 第四类「更新不生效」发生在问答之外，判据是另一条：先看对账认没认出这篇改了（内容指纹算之前有没有做换行归一化），再看缓存的 key 里有没有把索引版本和权限范围算进去。「什么时候必须失效」等价于「key 里有没有把那样东西算进去」，key 少放一样，那样东西变了缓存就不会失效。
  6. 可预期的追问：怎么让这套排查不靠人肉？答案是把分类做进评估面板——每一道没中的题自动标出它属于四类中的哪一类，并按租户分开统计。全局平均会把单个客户的塌方按人头摊薄，而线上会投诉的恰恰是那个客户。

  How to reason about it · think before answering

  1. This one is almost guaranteed to be asked, and most people answer with a flat list of possibilities: maybe chunking, maybe the prompt, maybe the model. A list is not triage. Triage means an order, a decision rule at each step, and each step eliminating half the search space.
  2. First decompose the complaint. 'Inaccurate' hides at least four distinct failures whose fixes do not transfer: off-topic answers, partial answers, misaligned citations, and stale content. So the first action is not to change a setting, it is to obtain the specific question and answer and classify it into one of those four.
  3. Then give the order along with its justification: read the pipeline right to left, fix it left to right. Right to left because the generated answer is what you see first; left to right because upstream errors are amplified downstream — no prompt can recover a document retrieval never fetched. Concretely: dump the candidate pool and the final context for that question, and check whether the answer document is in the pool at all. Absent means a retrieval debt; present but below the admission gate means a gate debt; admitted but never packed into the context budget means chunks too large or budget too small; all present and still unused means it is finally a generation problem.
  4. One detail worth volunteering because it is easy to get wrong: for multi-hop questions, diagnose the documents that are missing, not whether any one of them was retrieved. In our experiment one question needed two documents; the first ranked first every time and the second never entered the candidate pool at all. Judging by 'any of them' labels it a budget problem, and you can spend a full day tuning budgets to no effect. This distinction only occurs to someone who has actually triaged question by question.
  5. The fourth class, stale content, happens outside the question path and has its own rule: first check whether reconciliation even noticed the edit (was the content hash computed after line-ending normalization?), then check whether the cache key includes the index version and the permission scope. 'When must this expire' is equivalent to 'is that thing part of the key' — leave something out of the key and changes to it will never invalidate the entry.
  6. Expected follow-up: how do you stop relying on manual triage? Build the classification into the evaluation panel so every missed question is automatically labelled with one of the four classes, and report it per tenant. A global average dilutes one customer's collapse across the whole population, and that customer is exactly the one who will file the complaint.

  答题要点

  • 先把「答得不准」归类成四种病：答非所问、只答得出片段、引用错位、更新不生效——修法互不通用。
  • 排查从右往左看、修复从左往右修：先打印候选池与最终上下文，看答案文档卡在哪一层。
  • 四层判据依次是：没进候选池、进了没过门槛、过了没装进预算、都进了模型没用上。
  • 多跳题只诊断缺的那几篇；用「有没有捞到任意一篇」会把「根本没捞到」误判成预算问题。
  • 「更新不生效」查对账与缓存 key：什么时候必须失效，等价于 key 里有没有算进那样东西。

  Key points

  • Classify the complaint into four failures first — off-topic, partial, misaligned citation, stale — because their fixes do not transfer.
  • Read right to left, fix left to right: dump the candidate pool and final context and find which layer the answer document stalls at.
  • The four rules in order: never retrieved, retrieved but below the gate, admitted but squeezed out of the budget, packed but unused by the model.
  • For multi-hop, diagnose only the missing documents; judging by 'any one retrieved' mislabels a never-retrieved case as a budget problem.
  • For stale content, check reconciliation and the cache key: what must expire is exactly what the key must contain.
• 如果预算只够做三件事来提升一个已有 RAG 系统的效果，你选哪三件？为什么是这三件？If you could only fund three changes to improve an existing RAG system, which three would you pick and why those three?

  国内高频海外高频进阶#prioritization#evaluation#abstention

  分析过程 · 先想清楚再作答

  1. 这题在考优先级判断，而不是知识面。答成「上重排、上混合检索、上查询改写」这类手法清单几乎必然掉分——因为它跳过了一个前提：**你凭什么知道这三件对你的系统有用？** 面试官等的就是这句话。
  2. 所以第一件必须是**建评估**，而且理由要具体到不可反驳：没有秤，剩下两件做完你也说不清是变好还是变坏；有了秤，后面每一笔钱都能算回报。而且它便宜——检索侧三个指标是纯本地计算、几秒钟、零成本，能挂进每次提交；花时间的只是给题目标答案文档那一次。顺带说清评估集的配比：多跳与无答案各占一成以上，缺了无答案那一类，一个只会硬答的系统在报表上就是满分。
  3. 第二件是**把拒答从提示词搬进代码**，这一件的性价比通常最高而最容易被跳过。提示词里写十遍「找不到就说找不到」增益接近于零；而引用编号是一个闭集，判它存不存在只要一行代码，再加一道「这句话与被引块的实质重合度」就能拦住「编号是真的、内容是假的」那一类。我们实验里的基线拒答率是 0.0%——四道语料里根本没有答案的题一道都没闭嘴，这类缺陷在只报召回率的报表上完全不可见。
  4. 第三件要**先看失败案例再决定**，这才是这道题真正的答案。看完面板你会落到其中之一：多跳题占比高就补桥接检索或改索引结构；换个说法就捞不到，说明该上向量那一路或混合检索；答案捞到了却排不进上下文，那是重排或者预算的活。**先有失败案例，再有手法**——我们试过五种高级索引结构，没有一种跑赢基线，因为我们的系统压根没有那些结构要补的短板。
  5. 为什么不选那些看起来更亮的：Agentic 检索的收益集中在多跳题上而代价摊给全部问题；「全开」所有查询侧手法在我们的实测里召回率和默认配置一模一样，模型调用却是 2.5 倍、检索次数 4.3 倍。**堆手法很容易，说清楚为什么关掉某几项才是本事。**
  6. 可预期的追问：三件做完怎么证明钱花对了？答：每一项单独开关各跑一遍，报三笔账——指标涨了多少、延迟涨了多少、钱涨了多少。只报第一笔的提案不该被批准，包括你自己的。

  How to reason about it · think before answering

  1. This tests prioritisation, not breadth. Answering with a list of techniques — add reranking, add hybrid retrieval, add query rewriting — almost always loses points, because it skips a prerequisite: how do you know those three help your system? That is precisely the sentence the interviewer is waiting for.
  2. So the first item has to be building evaluation, with a reason specific enough to be unarguable: without a scale, you cannot tell whether the other two helped or hurt; with one, every subsequent spend has a measurable return. It is also cheap — the three retrieval metrics are pure local computation, run in seconds, cost nothing, and can gate every commit; the only real effort is labelling answer documents once. Include the composition rule: multi-hop and unanswerable each above ten percent, because without the unanswerable class a system that only ever guesses scores perfectly on your report.
  3. Second, move abstention out of the prompt and into code — usually the best return per unit of effort, and the item most often skipped. Writing 'say you don't know' ten times in a prompt buys almost nothing. Citation numbers are a closed set, so checking existence is one line, and adding a substantive-overlap check catches the harder forgery where the number is real but the content is not. Our baseline abstention rate was 0.0 percent: four questions with no answer in the corpus, zero of them declined — a defect that is completely invisible on a report that only shows recall.
  4. Third, look at the failure cases before deciding, which is the actual answer to this question. After reading the panel you land on one of a few branches: a high share of multi-hop means bridging retrieval or a different index structure; queries that miss when phrased differently mean you need the vector route or hybrid retrieval; answers retrieved but never packed into context means reranking or budget. Failure cases first, technique second — we tried five advanced index structures and not one beat the baseline, because our system simply did not have the weakness they address.
  5. Why not the flashier options: agentic retrieval concentrates its gains on multi-hop while spreading cost across every question, and in our measurements turning on every query-side technique produced exactly the same recall as the default configuration while using 2.5 times the model calls and 4.3 times the retrievals. Stacking techniques is easy; explaining why you switched several off is the skill.
  6. Expected follow-up: once the three are done, how do you prove the money was well spent? Toggle each one individually and report three ledgers — how much the metric moved, how much latency moved, how much cost moved. A proposal that reports only the first should not be approved, including your own.

  答题要点

  • 第一件是建评估：没有秤，另外两件做完也说不清变好还是变坏；检索侧指标零成本可挂进每次提交。
  • 评估集必须含无答案那一类，否则一个只会硬答的系统在报表上就是满分。
  • 第二件是把拒答从提示词搬进代码：编号是闭集，再加实质重合度就能拦住「编号真、内容假」。
  • 第三件由失败案例决定，不由手法清单决定——先有失败案例，再有索引结构或检索手法。
  • 每一项单独开关跑一遍并报三笔账：指标、延迟、钱。只报第一笔的提案不该被批准。

  Key points

  • First, build evaluation: without a scale the other two changes are unverifiable, and the retrieval metrics are cheap enough to gate every commit.
  • The golden set must include unanswerable questions, or a system that only ever guesses scores perfectly on your report.
  • Second, move abstention from the prompt into code: citation numbers are a closed set, and a substantive-overlap check catches real-number-fake-content forgeries.
  • Third is chosen by the failure cases, not by a list of techniques — failure cases first, index structure or retrieval trick second.
  • Toggle each change individually and report three ledgers: metric, latency, cost. A proposal reporting only the first should not be approved.