面试题库

14 天 RAG：从检索到可信回答

D1 为什么要检索：幻觉、知识截止与长上下文的代价，以及一个纯关键词的最小 RAG

• 什么时候该用检索增强生成，什么时候该微调，什么时候直接把文档塞进上下文就够了？When should you use retrieval-augmented generation, when should you fine-tune, and when is stuffing the documents into the context window good enough?

  国内高频海外高频基础#rag-basics#fine-tuning#long-context

  分析过程 · 先想清楚再作答

  1. 这题几乎每场都问，区分度不在能不能背出三条定义，而在你会不会给一条判据。只说「RAG 适合动态知识、微调适合特定风格」的人一抓一大把，面试官等的是下一句。
  2. 先给一条能当场套用的判据：模型缺的是「知道什么」还是「怎么说」。缺知识走检索，缺风格与输出格式走微调，这一刀切下去能分掉八成场景。
  3. 再拿三笔账把三条路排开：知识更新的代价（改文件立刻生效 / 重训以天计 / 改文件立刻生效）、单次成本（只付取回的几段 / 只付推理 / 每次都付全量材料）、能不能归因（能 / 不能 / 能但材料一多定位会飘）。
  4. 把上下文直塞的适用边界说清楚：材料总量小、更新不频繁、对单次成本不敏感的场景它最划算，因为工程量近乎为零。一旦材料涨到几百篇，或者同一批材料每天要被问上万次，成本曲线立刻反超。
  5. 最后主动补一句「什么时候都不该用检索」——任务的答案不依赖任何外部文档时（改写、翻译、格式转换），加检索只会引入噪声、延迟和成本。能主动划出不该用的边界，比会背适用场景更能证明你做过。
  6. 可预期的追问：能不能既微调又检索？答案是可以，而且常见——微调管输出格式与拒答口径，检索管事实，两者解决的不是同一个问题。

  How to reason about it · think before answering

  1. This question shows up in almost every loop. The differentiator is not reciting three definitions, it is offering a decision rule the interviewer can reuse.
  2. Lead with the rule: is the model missing knowledge, or missing a way of speaking? Missing knowledge means retrieval; missing style or output shape means fine-tuning. That single cut covers most cases.
  3. Then line up the three options against three costs: cost of updating knowledge, cost per request, and whether the answer can be traced back to a source. Retrieval updates by editing a file, fine-tuning takes a retraining cycle, and long-context pays for the whole corpus on every call.
  4. Give long-context its fair case: when the corpus is small, changes rarely, and request volume is low, stuffing it in is the cheapest engineering decision you can make. It stops being cheap once the corpus grows or the same material is queried thousands of times a day.
  5. Close by naming when none of this applies: if the answer does not depend on any external document (rewriting, translating, reformatting), retrieval only adds noise, latency and cost.
  6. Expected follow-up: can you do both? Yes, and it is common. Fine-tuning controls format and refusal behaviour, retrieval supplies the facts.

  答题要点

  • 一条判据：缺「知道什么」用检索，缺「怎么说」用微调。
  • 检索改文件即时生效、可归因、成本只跟取回的几段有关，代价是要自己建一套会出错的检索系统。
  • 微调擅长固化风格与输出格式，不擅长灌事实：数据一变就要重训，而且没法归因。
  • 长上下文直塞在小型、低频、少变的语料上最划算，材料变多或调用量变大之后成本与定位稳定性都会恶化。
  • 任务答案不依赖外部文档时三条路都不该用，直接调模型。

  Key points

  • One rule: retrieval for missing knowledge, fine-tuning for a missing way of speaking.
  • Retrieval updates instantly by editing files, supports citation, and costs scale with the retrieved passages rather than the corpus.
  • Fine-tuning is good at locking in style and output schema, poor at loading facts, and offers no traceability.
  • Long-context stuffing wins when the corpus is small, stable and queried infrequently; it loses on cost and on locating facts once the corpus grows.
  • If the answer does not depend on any document, use none of them.

D2 embedding 与向量检索：相似度、维度与模型选型，把文本存进 pgvector

• 余弦相似度和内积什么时候等价？如果向量没有归一化，用内积排序会出什么问题？When are cosine similarity and inner product equivalent? What goes wrong if you rank by inner product on vectors that are not normalised?

  国内高频海外高频基础#embeddings#similarity#normalisation

  分析过程 · 先想清楚再作答

  1. 这题是送分题，但区分度藏在后半句。只答「归一化之后两者等价」的人很多，面试官真正想听的是「没归一化会怎么坏」，因为那是线上真的会发生的事。
  2. 先把定义摆出来：余弦相似度等于内积除以两个向量模长的乘积。模长都是 1 时除数就是 1，所以余弦相似度就是内积——这一句话就是等价的全部理由，不需要额外的假设。
  3. 再说没归一化的后果：内积里混着「方向有多一致」和「向量有多长」两层信息。文本越长，模型输出的向量模长往往越大，于是排序会系统性地偏向长文档——这跟 BM25 里 b 参数要压的是同一个毛病，只是换了个地方冒出来。
  4. 点出这类 bug 的性质：它不报错。程序照常跑、结果照常出，只是名次悄悄偏了，你要跑一轮离线评估才可能发现。所以工程上的做法是在 embedding 的出口统一归一化一次，而不是靠每个调用点自觉。
  5. 补一句欧氏距离：向量都归一化之后，欧氏距离的平方等于 2 减去 2 倍内积，也就是余弦距离的单调函数，三种距离排出来的名次完全一致。这一句能说明你理解的是关系而不是三条并列的规则。
  6. 可预期的追问：那 pgvector 里该用哪个运算符？答案是既然已经归一化，`<=>`（余弦距离）和 `<#>`（负内积）名次一样，选 `<=>` 的理由是可读性和「就算哪天有人漏了归一化也不至于错」。

  How to reason about it · think before answering

  1. This starts as a giveaway, but the second half is where candidates separate. Many can say 'they are equivalent after normalisation'; few can describe what breaks without it.
  2. State the definition: cosine similarity is the inner product divided by the product of the two magnitudes. When both magnitudes are 1, the divisor is 1 and cosine reduces to the inner product. That is the whole argument.
  3. Then the failure mode: an un-normalised inner product mixes 'how aligned' with 'how long'. Longer texts tend to produce larger-magnitude vectors, so ranking drifts systematically toward long documents, the same bias BM25's b parameter exists to counter.
  4. Stress that this bug is silent. Nothing throws, results still look plausible, and only an offline evaluation reveals the drift. Hence the engineering rule: normalise once at the embedding boundary, never at each call site.
  5. Add Euclidean distance for completeness: on normalised vectors, squared L2 equals 2 minus twice the inner product, a monotone function of cosine distance, so all three metrics produce the same ranking.
  6. Expected follow-up: which pgvector operator should you use? Since the vectors are normalised, `<=>` and `<#>` rank identically; prefer `<=>` for readability and because it stays correct if someone later forgets to normalise.

  答题要点

  • 余弦相似度 = 内积 / 两个模长之积，模长为 1 时除数为 1，两者等价。
  • 没归一化时内积混入模长信息，长文档的向量模长普遍更大，排序会系统性偏向长文档。
  • 这类错误不报错，只能靠离线评估发现，所以要在 embed 出口统一归一化。
  • 归一化之后欧氏距离与余弦距离互为单调函数，三种运算符名次一致。
  • pgvector 里对应 `<->`（L2）、`<#>`（负内积）、`<=>`（余弦距离）三个运算符。

  Key points

  • Cosine equals inner product divided by both magnitudes; with unit magnitudes the divisor is 1, so they coincide.
  • Without normalisation the inner product carries magnitude, and longer documents usually have larger magnitudes, biasing the ranking.
  • The failure is silent, so normalise once at the embedding boundary and verify with offline evaluation.
  • On normalised vectors L2 and cosine are monotonically related, so all operators rank the same.
  • In pgvector the operators are `<->` for L2, `<#>` for negative inner product and `<=>` for cosine distance.

D3 文档进来这一关：PDF 与 HTML 解析、表格与扫描件、清洗规则和必须留下的元数据

• 为什么说解析质量决定了检索质量的上限？举一个具体的传导链条。Why is parsing quality the ceiling on retrieval quality? Walk through one concrete chain of propagation.

  国内高频海外高频基础#ingestion#data-quality#failure-analysis

  分析过程 · 先想清楚再作答

  1. 这是一道送分题，但很多人答成口号。判据只有一个：有没有给出一条能落到具体现象上的链条，而不是重复一遍「垃圾进垃圾出」。
  2. 先说清位置：解析在切块、建索引、检索、组装、生成这五环之前，是第零环。它的错误会被后面每一环放大，而且后面每一环都无法察觉——它们只是在忠实地处理一段已经错了的文字。
  3. 给一条具体链条：一张套餐配额表在 PDF 里丢了一列分隔符，抽出来串了行；切块照着错误的边界切，「专业版」和隔壁那一栏的值被切进同一块；索引把错误的词对记进倒排表；用户问「专业版存储配额多少」，这一块分数很高被排到第一；模型只依据给定材料回答，于是给出一个错误但带着正确引用编号的答案。
  4. 点破最要命的一句：这条链上没有任何一环会报错，回答甚至是带出处的，看起来比平时更可信。所以解析的错误不能靠事后发现，只能靠入口处的断言拦。
  5. 反过来说明「上限」二字：后面所有优化——向量、混合检索、重排、查询改写——优化的都是「从候选里挑得更准」。材料本身错了，挑得再准也是错的，所以它们的天花板由解析封死。
  6. 可预期的追问：那怎么证明是解析的锅？答案接回 D1 那条习惯——排查从右往左看，把检索出来的原文打印出来自己读一遍，如果原文本身就是串行的，那就不用再往生成侧查了。

  How to reason about it · think before answering

  1. This is a giveaway question that many people answer with a slogan. The only test is whether you produce a chain that lands on a concrete symptom instead of repeating garbage in, garbage out.
  2. Place it first: parsing sits before chunking, indexing, retrieval, context assembly and generation. Its errors are amplified by every later stage, and none of those stages can detect the problem because each is faithfully processing text that is already wrong.
  3. Give the chain: a pricing table in a PDF loses one column separator and comes out with cells shifted. Chunking splits on those wrong boundaries, so a plan name ends up next to the neighbouring column value. The index records the wrong term pairing. A user asks about that plan's storage quota, the corrupted chunk scores highest, and the model, faithfully answering only from the provided material, returns a wrong answer carrying a correct-looking citation.
  4. Name the nastiest part: nothing on that chain raises an error, and the answer even comes with a source, so it looks more trustworthy than usual. Parsing errors cannot be caught after the fact, only by assertions at ingest.
  5. Explain the word ceiling: every later optimisation, dense retrieval, hybrid search, reranking, query rewriting, improves how well you pick from the candidates. If the material itself is wrong, picking better still returns something wrong, so parsing caps all of them.
  6. Expected follow-up: how do you prove parsing is at fault? Reuse the habit from day one. Diagnose right to left and print the retrieved passages verbatim. If the source text is already scrambled, there is no point looking at the generation side.

  答题要点

  • 解析是五个环节之前的第零环，它的错误会被后面每一环放大，而后面每一环都察觉不到。
  • 具体链条：表格串行 → 切块按错误边界切 → 倒排表记进错误词对 → 检索把它排第一 → 模型据此给出带引用的错误答案。
  • 最危险的是全程零报错，且答案带着出处，看起来比平时更可信。
  • 后面所有优化解决的是「挑得更准」，材料本身错了就都无效，所以上限由解析封死。
  • 定位方法是排查从右往左：先把检索到的原文打印出来读一遍，原文错了就不必再查生成侧。

  Key points

  • Parsing is stage zero, before the five-stage pipeline; its errors are amplified downstream and invisible to every later stage.
  • Concrete chain: a shifted table, chunking on wrong boundaries, wrong term pairs in the index, that chunk ranked first, and a wrong answer delivered with a citation.
  • The dangerous part is that nothing errors out and the answer carries a source, so it looks more credible than usual.
  • Later techniques only improve selection from candidates; if the material is wrong, better selection still returns something wrong.
  • Diagnose right to left: print the retrieved passages first, and if the source text is already broken, stop looking at the generation side.

D4 切块策略：固定、递归、按结构、父子与语义五种切法，以及用评估而不是直觉来选

• 重叠区设成块长的百分之多少合适？重叠过大会带来什么具体问题？What overlap ratio would you use, and what concretely goes wrong when the overlap is too large?

  国内高频海外高频基础#chunking#overlap

  分析过程 · 先想清楚再作答

  1. 这是一道送分题，但送分点不在那个百分比上，而在后半句。只答「一般一到两成」就停住的人，面试官会认为他没跑过。
  2. 先说清重叠在补救什么：固定长度切法会把句子从中间切开，重叠让被切开的那句话至少在相邻两块之一里是完整的。它是给「乱切」打的补丁，不是一个独立的优化。
  3. 由此推出第一个结论：如果你用的是按结构切或递归切，边界本来就落在语义位置上，重叠的必要性会大幅下降，甚至可以是零。**重叠比例这个问题的前提是切法**，脱开切法谈比例就是背数字。
  4. 过大的代价要说三笔，越具体越好。存储与 token：块长 400、重叠从 0 加到 80，索引 token 会涨一成半左右，这笔钱在向量库是存储费、在检索时是比对量。
  5. 检索冗余：相邻块越像，前几名越可能是同一段话的三个版本，你以为给了模型三条证据，其实是一条说了三遍。这一条在重排之前基本无解。
  6. 引用定位：同一句话出现在两个块里，模型标出处该标哪一个，这会直接变成引用校验环节要处理的边界情况。可预期的追问就是「那你怎么去重」，答按内容指纹或最长公共子串在结果层合并，而不是在切块层想办法。

  How to reason about it · think before answering

  1. This is a giveaway question, but the marks are in the second half, not the percentage. Stopping at 'usually ten to twenty percent' reads like someone who has never run it.
  2. Say what overlap is patching: fixed-length splitting cuts sentences in half, and overlap guarantees the broken sentence survives intact in at least one of the two neighbours. It is a patch for careless splitting, not an optimisation of its own.
  3. That yields the first conclusion: with structural or recursive splitting the boundaries already land on semantic positions, so the need for overlap drops sharply and can legitimately be zero. The ratio question is meaningless without naming the strategy.
  4. Give three concrete costs. Storage and tokens: at 400-character chunks, moving overlap from 0 to 80 grows total index tokens by roughly fifteen percent, which is storage cost in the vector store and comparison work at query time.
  5. Retrieval redundancy: the more neighbours overlap, the more likely the top results are three versions of the same passage. You think you handed the model three pieces of evidence; you handed it one, three times. Nothing fixes this before reranking.
  6. Citation resolution: when a sentence lives in two chunks, which one does the model cite. Expect the follow-up on deduplication: merge at the result layer using a content fingerprint or longest common substring, not by tweaking the chunker.

  答题要点

  • 经验区间是块长的一到两成，但这个数字的前提是你用的是固定长度切法。
  • 按结构或递归切时边界本来就在语义位置上，重叠可以很小甚至为零。
  • 过大代价一：索引 token 与存储明显上涨，块长 400 时重叠加到 80 大约涨一成半。
  • 过大代价二：相邻块高度相似，检索前几名变成同一段话的多个版本，证据多样性是假的。
  • 过大代价三：同一句话跨块出现，引用标注和去重都要额外处理。

  Key points

  • Ten to twenty percent of chunk length is the working range, but that number assumes fixed-length splitting.
  • With structural or recursive splitting the boundaries are already semantic, so overlap can be small or zero.
  • Cost one: index tokens and storage grow noticeably; at 400-character chunks, an 80-character overlap adds roughly fifteen percent.
  • Cost two: neighbouring chunks become near-duplicates, so the top results are several versions of one passage and the evidence diversity is illusory.
  • Cost three: a sentence spanning two chunks complicates citation attribution and forces result-level deduplication.

D6 生成这一侧：上下文怎么排、引用怎么标、什么时候必须拒答，以及流式回答

• 上下文里材料的排列顺序会影响回答质量吗？如果会，你会怎么排？Does the ordering of retrieved passages in the context affect answer quality? If so, how would you order them?

  国内高频海外高频基础#context-assembly#prompt-engineering#ordering

  分析过程 · 先想清楚再作答

  1. 这是一道送分题，但答成「按相关性从高到低排」就只拿到一半分。面试官想听的是你知不知道位置本身是个变量。
  2. 结论先说：会影响。模型对上下文开头和结尾的材料明显更敏感，正中间的最容易被读漏。所以简单按分数从高到低顺排，等于把第二重要的材料放进了最不容易被读到的位置。
  3. 给出排法：第 1 名放开头、第 2 名放结尾、第 3 名放第二位、第 4 名放倒数第二位，依次往里收。这样按分数排下来越靠中间的块本来就越不重要，被读漏的代价最小。
  4. 顺带把排序之外的三道手续说全，显得你真的写过这段代码：同分要有决胜键（否则块编号会在两次运行之间飘，日志对不上）、要按归一化文本去重（同一段话常在手册和问答里各出现一次）、要有 token 预算并且塞不下时不要直接停。
  5. 可预期的追问：这个结论怎么验证？答案是别猜——固定一批问题，只改排列顺序跑对照，看指标差多少。位置效应在不同模型、不同上下文长度上强弱不一样，把它当成一个要在自己数据上量的参数，而不是一条普适定律。

  How to reason about it · think before answering

  1. This is a warm-up question, but 'sort by relevance descending' only earns half the credit. The interviewer wants to know whether you treat position itself as a variable.
  2. State the conclusion first: it does matter. Models attend more reliably to material at the start and the end of the context, and are most likely to miss what sits in the middle. Plain descending order therefore parks your second-best passage in the worst spot.
  3. Give the ordering: rank one first, rank two last, rank three second, rank four second-to-last, folding inward. Whatever ends up in the middle is by construction the least important, so the cost of it being skipped is smallest.
  4. Round it out with the other assembly steps, which shows you have written this code: a deterministic tiebreaker (otherwise block numbers drift between runs and your logs stop matching), dedupe on normalised text, and a token budget that skips rather than stops when a block does not fit.
  5. Expected follow-up: how would you verify this? Do not guess. Hold the question set fixed, vary only the ordering, and measure. Position effects differ by model and context length, so treat it as a parameter to measure on your own data rather than a universal law.

  答题要点

  • 会影响：开头和结尾的材料更容易被用上，正中间的最容易被读漏。
  • 排法是最重要的放两端：第 1 名开头、第 2 名结尾、第 3 名第二位，依次往里收。
  • 组装还要做三件事：同分给决胜键保证编号稳定、按归一化文本去重、控 token 预算且塞不下时跳过而不是终止。
  • 位置效应的强弱因模型与上下文长度而异，要在自己的数据上做对照实验量出来，不能当普适定律照搬。

  Key points

  • Yes: material at the head and tail is used more reliably, the middle is most often skipped.
  • Put the strongest at both ends: rank one first, rank two last, rank three second, folding inward.
  • Assembly also needs a deterministic tiebreaker for stable numbering, dedupe on normalised text, and a token budget that skips oversized blocks instead of stopping.
  • The strength of the effect varies by model and context length, so measure it on your own data instead of quoting it as a law.

D7 第一周综合：把六天的零件装成一个可一键启动的检索问答服务并复盘

• 你会怎么划分一个检索增强生成系统的模块边界？其中哪一层最应该做成可替换的，为什么？How would you draw the module boundaries of a RAG system, and which layer most needs to be swappable? Why?

  国内高频海外高频基础#architecture#modularity#embeddings

  分析过程 · 先想清楚再作答

  1. 这题考的是你有没有真的维护过这类系统。只按「解析、切块、检索、生成」复述一遍流程图，面试官会判定你只搭过 demo——流程图人人都会画，切口画在哪才是经验。
  2. 给一条可复用的判据再往下推：切口应该落在「将来最可能被整个换掉」的地方，而不是按代码量或者功能名称均分。
  3. 用它过一遍：embedding 一年会换好几次，换一次库里所有向量作废、必须全量重算，所以它必须是接口；存储可能从 PostgreSQL 换成专用向量库，而且摄取和查询都要通过它，所以它是两条链路的唯一交界；切块策略在调优期天天改，所以它必须是配置项而不是硬编码。
  4. 结论：最该做成可替换的是 embedding 那一层，理由不是「设计模式」，而是「换模型这件事真的会发生，且发生时代价极高」。
  5. 顺手点出抽象的代价：每多一层间接就多一次跳转和一份心智负担，所以判据是「那件事会不会真的发生」，不会发生的别抽象。
  6. 可预期的追问：那生成模型要不要也抽象？答案是要，但优先级低——换生成模型不需要重算任何存量数据，回滚也便宜，所以它是配置项而不是一层接口。

  How to reason about it · think before answering

  1. This question separates people who have maintained such a system from people who have only built a demo. Reciting the pipeline diagram is not an answer; where you cut it is.
  2. Offer a reusable criterion first: cut where a layer is most likely to be replaced wholesale, not by lines of code or by tidy functional names.
  3. Apply it. Embedding models change several times a year, and each change invalidates every stored vector, so that layer must be an interface. Storage may move from PostgreSQL to a dedicated vector database, and both ingestion and query talk through it, so it is the single shared boundary. Chunking changes daily during tuning, so it belongs in config, not in code.
  4. Conclusion: the embedding layer is the one that must be swappable, because the swap is both likely and expensive, not because interfaces are good style.
  5. Name the cost of abstraction too: every indirection is one more hop while debugging, so the test is whether the change will actually happen.
  6. Expected follow-up: should the generation model be abstracted as well? Yes, but at lower priority, because swapping it does not force recomputation of stored data and rollback is cheap. It is a config value, not a layer.

  答题要点

  • 先给判据：切口落在最可能被整体替换的那一层，不按代码量或功能名称均分。
  • embedding 是最该抽象的一层：换模型意味着存量向量全部作废、必须全量重算，代价高且真的会发生。
  • 存储层是摄取与查询唯一的交界，接口要先定下来再谈两边实现。
  • 切块与检索路数做成配置项，因为它们在调优期改动最频繁，改一次不该动代码。
  • 抽象有成本，判据是那件事会不会真的发生；不会发生的抽象就是过度设计。

  Key points

  • Lead with the criterion: cut where a layer is most likely to be replaced wholesale.
  • The embedding layer is the one to abstract: swapping models invalidates every stored vector and forces a full recompute.
  • Storage is the single boundary shared by ingestion and query, so define its interface before either implementation.
  • Chunking and retrieval routes belong in configuration because they change most often during tuning.
  • Abstraction costs indirection, so only abstract changes that will actually happen.

D8 评估先行：搭 golden set、算召回与排序指标、用模型当裁判判忠实度

• RAG 的评估集里为什么一定要放语料里没有答案的问题？不放会掩盖什么？Why must a RAG evaluation set include questions the corpus cannot answer, and what does leaving them out hide?

  国内高频海外高频基础#evaluation#abstention#golden-set

  分析过程 · 先想清楚再作答

  1. 这题看着简单，实际是在问「你有没有想过评估集本身也会说谎」。答成「为了测试拒答功能」只算及格，答出「不放会让某个故障在报表上完全不可见」才是满分。
  2. 推导只有一步：一个只会硬答的系统，在只有可答问题的评估集上能拿到很高的分——它每次都塞材料给模型，模型每次都编一段话，而评估集根本没有「应该拒答」这一栏。于是最危险的故障在报表上是不存在的。
  3. 结论：无答案问题是唯一能让「乱编」显形的东西。它不参与召回率，它的指标是拒答率——检索侧有没有把不够格的候选全挡下来，生成侧有没有真的说出「资料里没有」。
  4. 出题上有个必须说的细节：无答案问题必须留强干扰词，比如问「网页端支持哪些浏览器」而语料里恰好有一句「提交工单请附上浏览器与版本」。没有干扰词的无答案题检索器一条都捞不到，你测出来的是分词器不是系统。
  5. 可预期的追问是「拒答率低怎么办」。分两层查：先看检索侧的门槛是不是形同虚设（分数阈值定得太低，不相干的块也过关），再看生成侧的提示词有没有明确的拒答指令，两层都要有，只靠提示词兜是不牢的。

  How to reason about it · think before answering

  1. It looks easy but really asks whether you have considered that the eval set itself can lie. 'To test the refusal path' is a pass; 'without them the worst failure is invisible in the report' is a full mark.
  2. The derivation is one step: a system that always answers scores well on a set of answerable questions only. It stuffs context in, the model writes something, and the set has no column for 'should have refused'. The most dangerous failure simply does not appear.
  3. Conclusion: unanswerable questions are the only thing that makes fabrication visible. They are excluded from recall and scored on abstention instead - did retrieval gate out every weak candidate, and did generation actually say the material does not cover this.
  4. One authoring detail worth stating: unanswerable questions need strong distractor terms. Ask which browsers the web client supports when the corpus only says 'attach your browser and version when filing a ticket'. Without distractors retrieval returns nothing and you are testing your tokenizer, not your system.
  5. Expected follow-up: what if the abstention rate is low? Check two layers - whether the retrieval score gate is effectively a no-op, and whether the generation prompt carries an explicit refusal instruction. You need both; a prompt alone is not a reliable gate.

  答题要点

  • 只有可答问题的评估集，会让「不知道也硬答」这个故障完全不可见。
  • 无答案问题不算召回率，它的指标是拒答率，检索侧和生成侧各看一层。
  • 出题必须留强干扰词，否则检索器一条都捞不到，测的是分词器。
  • 建议无答案题占比不低于评估集的一成五，跟多跳题一起构成覆盖度底线。
  • 拒答率低要分两层查：检索门槛是否形同虚设，生成提示词有没有拒答指令。

  Key points

  • An all-answerable eval set makes 'answers confidently when it should not' completely invisible.
  • Unanswerable items are scored on abstention, not recall, and you check both the retrieval gate and the generation refusal.
  • Author them with strong distractor terms, or retrieval returns nothing and you are testing the tokenizer.
  • Keep them at roughly 15% or more of the set, alongside multi-hop items, as the coverage floor.
  • A low abstention rate splits into two causes: a no-op retrieval score gate, or a missing refusal instruction in the prompt.

D9 混合检索与重排：两路召回、倒数排名融合，再用交叉编码器把前几名重新排一遍

• 交叉编码器为什么比双编码器准？既然更准，为什么不干脆拿它直接检索全库？Why is a cross-encoder more accurate than a bi-encoder? And if it is more accurate, why not just use it to search the whole corpus directly?

  国内高频海外高频基础#cross-encoder#bi-encoder

  分析过程 · 先想清楚再作答

  1. 这是一道送分题，但送分题的区分度在第二问。只答「交叉编码器慢」是不够的，要说清慢在结构上的哪一处。
  2. 先给结构差异：双编码器把查询和文档**各自**编码成向量，两者从头到尾没有见过面，最后只靠一次内积凑到一起；交叉编码器把查询和文档拼成一段文本一起过模型，每一层注意力都能让查询的词去看文档的词。
  3. 由此推出准确率差异的来源：双编码器要把一篇文档压成一个固定长度的向量，压缩必然丢信息，「周敏是平台组组长」里两个词的绑定关系未必留得下来；交叉编码器不压缩，它当场对齐。
  4. 第二问的答案就藏在同一个结构里：双编码器的文档向量**可以离线算好**，查询时只做向量检索；交叉编码器没有任何东西能预先算好，N 篇文档就要跑 N 次前向。十万块的语料重排一遍，等于每次提问都把整个库过一遍模型。
  5. 所以工程上的定位是分工：召回负责在全库里捞出一小批（便宜、可索引），重排负责把这一小批的顺序改对（贵、准）。默认只重排融合后的前 20 条。
  6. 可预期的追问：有没有中间路线？答有——后期交互（late interaction）那一类，文档侧提前算好词级表示、查询侧当场做交互，精度和成本都在两者之间，代价是索引体积大得多。

  How to reason about it · think before answering

  1. This is a giveaway question, but the discriminating half is the second part. Saying `cross-encoders are slow` is not enough; you have to point at the structural reason.
  2. Start with the structure: a bi-encoder encodes query and document **separately** into vectors that never meet until a single dot product at the end; a cross-encoder concatenates query and document into one sequence, so every attention layer lets query tokens attend to document tokens.
  3. That yields the accuracy gap: a bi-encoder must compress a document into one fixed-length vector, and compression loses information — the binding between `Zhou Min` and `platform team lead` may not survive. A cross-encoder does not compress; it aligns them on the spot.
  4. The answer to the second half hides in the same structure: bi-encoder document vectors can be computed **offline** and indexed, so query time is just a vector search. A cross-encoder has nothing to precompute — N documents means N forward passes. Reranking a 100k-chunk corpus means pushing the entire corpus through a model on every question.
  5. So the engineering split is a division of labor: recall pulls a small batch out of the whole corpus (cheap, indexable), reranking fixes the order of that batch (expensive, accurate). The default is to rerank only the top 20 after fusion.
  6. Expected follow-up: is there a middle path? Yes — late interaction, where token-level document representations are precomputed and the interaction happens at query time. Accuracy and cost land between the two, at the price of a much larger index.

  答题要点

  • 双编码器各自编码、最后一次内积；交叉编码器把查询和文档拼在一起过模型，注意力可以跨两者对齐。
  • 准确率差异来自压缩：双编码器把整篇文档压成一个向量，绑定关系会丢；交叉编码器不压缩。
  • 双编码器的文档向量能离线算好并建索引，交叉编码器没有任何东西可以预先算好。
  • 全库重排等于每次提问把整个语料过一遍模型，成本随语料规模线性增长。
  • 标准分工是召回加重排，重排只作用于融合后的前几十条。

  Key points

  • A bi-encoder encodes both sides separately and joins them with one dot product; a cross-encoder concatenates them so attention can align across the pair.
  • The accuracy gap comes from compression: a bi-encoder squeezes a whole document into one vector and loses bindings; a cross-encoder does not compress.
  • Bi-encoder document vectors can be computed offline and indexed; a cross-encoder has nothing to precompute.
  • Reranking the full corpus means running every chunk through a model on every question, so cost scales linearly with corpus size.
  • The standard split is recall plus rerank, with reranking applied only to the top few dozen after fusion.

D10 查询侧优化：改写、假设文档嵌入、多路查询、后退提问与意图路由

• 多轮对话里怎么处理指代？不做指代消解最典型的翻车场景是什么？How do you handle coreference in multi-turn RAG, and what is the classic failure when you skip it?

  国内高频海外高频基础#coreference#multi-turn#query-rewriting

  分析过程 · 先想清楚再作答

  1. 这是一道送分题，但送分题也有高下之分：只说「把历史拼进查询里」的答案，会被追问一句「历史越拼越长怎么办」就卡住。
  2. 先把做法说清：在检索之前加一次很短的改写调用，输入是最近几轮对话加本轮问题，输出是一行可以直接检索的检索式；温度设 0 保证同一句话每次改成同一个结果，并在提示词里明确禁止模型顺手回答问题。
  3. 为什么不是「把历史整个拼进查询」：历史越拼越长，噪声词把逆文档频率摊薄，检索反而更差；而且历史里包含上一轮的答案，等于拿答案去检索答案。改写的产出是一句话，不是一段历史。
  4. 最典型的翻车场景要举实例：上一轮问「这个操作必须由谁审批」，答「必须由某某组组长审批」；这一轮问「这个人叫什么名字」。不消解直接检索这七个字，实测是**一条候选都过不了门槛，系统只能拒答**。注意失败方式不是答错，是「明明语料里有答案却说找不到」，用户体验是崩塌式的。
  5. 补一条顺序上的坑：改写必须在意图路由**之前**。「这个人」是典型的多跳信号词，路由看到它会判成多跳、白跑一轮；改写之后它只是个普通单跳问题。同理，多路查询、后退提问也都要建立在改写后的那句话上，否则错误被放大好几倍。
  6. 可预期的追问是「怎么知道要不要改写」。答：短问题、含指代词、含省略（「那审计日志呢」）时才触发，纯新话题跳过——这一步很便宜，但能省掉一大半调用。

  How to reason about it · think before answering

  1. This is a warm-up question, but there is still a gap between answers. Saying "just concatenate the history into the query" invites a follow-up about growing histories that most candidates cannot handle.
  2. State the mechanism: insert a short rewrite call before retrieval that takes the last few turns plus the current question and returns one retrieval-ready line. Set temperature to 0 so the same input always yields the same query, and forbid the model from answering the question in the prompt.
  3. Explain why concatenation is worse: history grows without bound, filler words dilute inverse document frequency, and the previous answer leaks in — you end up retrieving an answer with an answer. The rewriter emits one sentence, not a transcript.
  4. Make the failure concrete. Turn one: "who must sign off on this operation?" Answer: "the platform team lead." Turn two: "what is that person's name?" Retrieved unresolved, not a single candidate clears the admission gate and the system refuses — even though the corpus contains the answer. The failure is not a wrong answer, it is a false "not found" right after the user's own question.
  5. Add the ordering trap: rewrite before intent routing. A pronoun is a classic multi-hop signal, so an unresolved query gets routed to the expensive path for nothing; after rewriting it is an ordinary single-hop question. Multi-query and step-back must also sit downstream of the rewrite, or one unresolved pronoun becomes three.
  6. Expect "how do you decide when to rewrite?" Trigger on short queries, pronouns and elliptical follow-ups; skip on a clearly new topic. The check is nearly free and removes most of the calls.

  答题要点

  • 在检索前加一次短改写调用，输入最近几轮加本轮问题，输出一行检索式，温度 0，禁止模型回答问题。
  • 不要把历史整段拼进查询：越拼越长、噪声稀释逆文档频率，还会拿上一轮的答案去检索。
  • 典型翻车：上一轮的「这个人 / 他 / 那个」不消解，检索一条都过不了门槛，系统在有答案的情况下拒答。
  • 失败方式是「假的查不到」，比答错更伤体验，因为用户刚刚才问过同一件事。
  • 顺序：先改写、再路由，多路查询与后退提问都建立在改写后的查询上。

  Key points

  • Add a short rewrite call before retrieval: last few turns plus current question in, one retrieval line out, temperature 0, answering explicitly forbidden.
  • Do not splice the whole history into the query — it grows unbounded, dilutes IDF, and leaks the previous answer into the search.
  • Classic failure: an unresolved pronoun means no candidate clears the gate, so the system refuses a question the corpus can answer.
  • That false "not found" hurts more than a wrong answer, since the user just asked about the same thing.
  • Order matters: rewrite first, then route; multi-query and step-back both build on the rewritten query.

D12 Agentic RAG：把检索做成工具，让模型自己决定查不查、查几次、要不要推翻重来

• 把检索包成一个工具交给模型，这个工具的描述该怎么写？写不好会导致哪些具体的错误行为？You are exposing retrieval to a model as a tool. How do you write the tool description, and what concrete failure modes appear when you write it badly?

  国内高频海外高频基础#tool-design#agentic-rag#prompting

  分析过程 · 先想清楚再作答

  1. 这题的题眼是「具体的错误行为」。只会背「描述要写清楚工具的用途」的，一句话就暴露了没上过线——面试官想听的是描述里少一句话，线上就多一类工单。
  2. 先给结构：一段合格的工具描述要回答四件事——库里有什么和没有什么、什么时候必须用、什么时候不要用、查询串写成什么形状。四条各对应一类事故，逐条挂钩着说最有说服力。
  3. 逐条挂钩：不写范围，模型拿它当搜索引擎，问天气也去查；不写「必须用」，涉及公司制度的问题被模型凭记忆编答案，而且编得非常像真的；不写「不要用」，闲聊和翻译都触发一次无谓检索，成本和延迟白涨；不写查询形状，模型把用户整句问话塞进 query，「叫什么名字」这种疑问词进了检索，纯噪声。
  4. 最后一条最值钱也最容易漏：在描述里加一句「写成关键词短语，不要带疑问词」，比在检索侧做十种查询清洗都管用——问题在源头，就在源头修。
  5. 补一个生产视角：参数里的过滤字段（比如部门）要写明「只在确定时才填」。模型倾向于把可选参数填满，填错一个部门就把正确答案挡在库外，而这种错误在日志里看不出来，表现是「检索没结果」。
  6. 可预期的追问是「怎么验证描述写对了」。答案是拿一批负样本跑：闲聊、翻译、算术、以及答案已在对话里的追问，看模型有没有多调一次工具；这类回归是能自动化的。

  How to reason about it · think before answering

  1. The discriminator is whether you can name concrete failure modes. Reciting 'the description should be clear' signals you have never shipped one.
  2. Give the structure first: a usable description answers four things - what is and is not in the corpus, when the tool must be called, when it must not be called, and what shape the query string should take.
  3. Attach a failure to each: no scope and the model treats it as a web search; no 'must call' and it answers policy questions from memory, convincingly; no 'must not call' and greetings or translations each burn a retrieval; no query shape and the model pastes the raw user sentence in, dragging interrogative words into the index.
  4. The query-shape line is the cheapest win: one sentence saying 'keyword phrase, no question words' beats ten heuristics for query cleaning on the retrieval side.
  5. Production angle: optional filter parameters such as department need an explicit 'only set this when you are certain'. Models like to fill optional fields, and a wrong filter hides the correct answer while the logs only show 'no results'.
  6. Expected follow-up: how do you verify the description works? Run a negative suite - small talk, translation, arithmetic, follow-ups already answered in the conversation - and assert the tool was not called. That regression is automatable.

  答题要点

  • 描述是写给模型看的提示词，不是注释；四段式：范围、什么时候用、什么时候不用、查询写成什么形状。
  • 不写范围会被当成搜索引擎；不写「必须用」会导致凭记忆编答案。
  • 不写「不要用」会让闲聊也触发检索，成本和延迟白涨。
  • 写明查询要用关键词短语、不带疑问词，比在检索侧清洗查询更根本。
  • 可选过滤参数要写「只在确定时才填」，填错会静默地把正确答案挡在外面。
  • 用一批负样本（闲聊、翻译、算术）做回归，断言工具没有被调用。

  Key points

  • The description is a prompt for the model, not a code comment: scope, when to call, when not to call, query shape.
  • Missing scope turns it into a web search; missing 'must call' produces confident answers from memory.
  • Missing 'do not call' makes small talk trigger retrieval, paying cost and latency for nothing.
  • Stating 'keyword phrase, no question words' fixes query pollution at the source.
  • Optional filters need 'only set when certain' - a wrong filter silently hides the right answer.
  • Regression-test with a negative suite and assert the tool was not invoked.

D14 综合项目与复盘：多租户企业知识库问答，一张 RAG 决策地图与面试专题

• 怎么向不懂技术的业务方证明你的检索系统真的变好了？How do you convince a non-technical stakeholder that your retrieval system actually got better?

  国内高频海外高频基础#evaluation#stakeholder-communication#abstention

  分析过程 · 先想清楚再作答

  1. 这题在考沟通，但拿分点在技术判断上：你选哪几个数字给业务方看，暴露了你自己有没有看懂这些指标。把召回率、nDCG、MRR 一股脑摊出去的答法会被判成不懂受众；只说「用户反馈变好了」又会被判成没有度量。
  2. 先立一条原则：**给业务方看的必须是他们能自己判断对错的东西**。归一化折损累计增益他们没法判断，而「这一百个真实问题里，系统答对了多少、答错了多少、老老实实说查不到了多少」他们一眼就能判断。所以对外的口径应该是三个数：答对率、答错率、拒答率，而且三个加起来是一百。
  3. 关键是把**答错和拒答分开**。这一条最能建立信任：查不到就说查不到不是故障，是正确输出；真正的故障是查不到还编一段。很多团队只报「准确率」，结果一个学会了一直拒答的系统能刷出满分——所以这三个数必须并排出现，缺一个都能被骗。
  4. 然后给可核对的证据，而不是只给数字：**挑十条真实问题做前后对照**，各贴出改动前和改动后的回答，每句结论后面挂着可点开的引用。业务方点开原文核对一遍，比看任何百分比都有说服力，而且这个动作顺带完成了一次人工抽检——你自己也需要它来校准模型裁判靠不靠谱。
  5. 本课里有一条要主动说的教训：**一列全是满分说明尺子坏了**。我们的题目是从语料反向出的，字面重合度过高，平均倒数排名恒为 1.0000。这个数字拿给业务方看，只会换来一次「那你们已经完美了」的误会，而它其实是指标饱和。指标撞天花板时该做的是把题目出难一点。
  6. 可预期的追问：那怎么让业务方参与进来？答一条很实用的：让他们提供题目。把线上答错的问题一条条补进标准答案集，评估集是长出来的，而不是一次性造好的；这样每一次改进都能指着「你上次提的那个问题现在答对了」，比任何汇报都直接。

  How to reason about it · think before answering

  1. This is a communication question whose scoring hinges on technical judgement: which numbers you choose to show reveals whether you understand the metrics yourself. Dumping recall, nDCG and MRR on a business stakeholder reads as tone-deaf; saying 'user feedback improved' reads as unmeasured.
  2. Start from a principle: show them something they can adjudicate themselves. They cannot judge normalized discounted cumulative gain, but they can absolutely judge 'out of these hundred real questions, how many did it answer correctly, how many wrongly, and how many did it honestly decline'. So the external framing is three numbers — correct, wrong, declined — and they sum to one hundred.
  3. The crucial move is separating wrong from declined, and it is the fastest way to earn trust: saying 'not found' is a correct output, not a failure; the failure is inventing an answer when nothing was found. Teams that report a single 'accuracy' number can be gamed by a system that learns to decline everything, which is why all three must appear side by side.
  4. Then supply checkable evidence rather than only numbers: take ten real questions and show before-and-after answers with clickable citations on every claim. A stakeholder who opens the source and verifies one claim is more convinced than by any percentage, and the exercise doubles as the human spot-check you need anyway to calibrate whether your model judge is trustworthy.
  5. There is a lesson from this course worth volunteering: a column of perfect scores means the ruler is broken. Our questions were written backwards from the corpus, lexical overlap is unusually high, and mean reciprocal rank sits at exactly 1.0000. Showing that to a stakeholder only invites the misreading that you are already perfect, when in fact the metric has saturated. When a metric hits the ceiling, the response is to make the questions harder.
  6. Expected follow-up: how do you get the business side involved? One very practical answer: let them supply questions. Every production miss gets appended to the golden set, so the evaluation set grows rather than being built once. Then each release can point at 'the question you raised last month now answers correctly', which lands better than any status report.

  答题要点

  • 对外只用三个他们能自己判断的数：答对率、答错率、拒答率，三者相加为一百。
  • 答错和拒答必须分开——查不到就说查不到是正确输出，只报一个准确率会被「一直拒答」刷满分。
  • 配十条真实问题的前后对照，每句结论挂可点开的引用，让他们自己核对原文。
  • 主动说明指标饱和：某一列恒为满分是尺子坏了，不是系统完美，该做的是把题目出难一点。
  • 让业务方提供题目，把线上答错的问题补进标准答案集——评估集是长出来的。

  Key points

  • Externally report three numbers they can adjudicate: correct, wrong, declined — summing to one hundred.
  • Keep wrong and declined separate; a single accuracy number is gamed by a system that learns to decline everything.
  • Pair it with ten before-and-after real questions, every claim carrying a citation they can open and verify.
  • Volunteer the saturation caveat: a column of perfect scores means a broken ruler, and the fix is harder questions.
  • Let stakeholders contribute questions; append every production miss to the golden set so it grows over time.