面试题库

14 天 RAG：从检索到可信回答

D3 文档进来这一关：PDF 与 HTML 解析、表格与扫描件、清洗规则和必须留下的元数据

• 扫描件走光学字符识别之后错字率不低，这些噪声会怎样影响检索和生成？怎么缓解？OCR output from scanned documents carries a non-trivial error rate. How does that noise propagate into retrieval and generation, and how do you mitigate it?

  国内高频海外高频深入#ocr#data-quality#hybrid-search

  分析过程 · 先想清楚再作答

  1. 这题考的是你会不会顺着链条推传导，而不是背「OCR 会有错字」这句废话。判据是有没有分别说清「检索侧怎么错」和「生成侧怎么错」——它们的失效方式完全不同。
  2. 先说检索侧。中文 OCR 的错主要是形近字，「已」认成「己」、「板」认成「版」。关键词检索是字面匹配，一个字错了这个词就查不到；更隐蔽的是二元组分词会连带毁掉相邻两个词元，一个错字影响的其实是两处。这一路的表现是召回悄悄掉下去，而且不报错。
  3. 再说生成侧。错字进了上下文，模型往往能读懂大意，但一旦是关键实体（人名、型号、金额、日期）出错，它会照着错的答，而且答得很自信。更麻烦的是引用校验也会跟着失效——原文本身就是错的，校验通过了也没意义。
  4. 缓解按三层说。入口层：先用空文本比例这类断言判断这份 PDF 有没有文本层，有就别走 OCR；真要走，保留原图链接以便人工复核。
  5. 检索层：靠混合检索兜底，向量一路对个别错字不敏感，能补上关键词一路的失手，这是 D9 那套东西在这里的具体价值。生成层：把低置信度的页面标出来，让模型在引用它们时明确提示「该材料来自扫描件，可能有识别误差」。
  6. 可预期的追问：能不能自动纠错？可以但要克制——用词典或模型做后处理会修好一批，也会「修」坏一批原本正确的专有名词。稳妥的做法是只对置信度低的片段做纠错，并且保留原文以便回退。

  How to reason about it · think before answering

  1. This tests whether you can trace propagation rather than recite that OCR makes mistakes. The differentiator is separating how retrieval fails from how generation fails, because the two failure modes are entirely different.
  2. Retrieval first. Chinese OCR errors are mostly visually similar characters. Keyword search is literal, so one wrong character makes the term unmatchable, and bigram tokenisation makes it worse because a single wrong character corrupts two adjacent tokens. Recall drops quietly and nothing raises an error.
  3. Generation second. The model usually reads through minor noise, but when the corrupted token is a key entity such as a name, a model number, an amount or a date, it answers confidently with the wrong value. Citation checking degrades too: verifying against a source that is itself wrong proves nothing.
  4. Mitigate in three layers. At ingest, use an empty-text assertion to decide whether the PDF even needs OCR, and keep a link to the original image so a human can verify.
  5. At retrieval, hybrid search absorbs some of the damage because dense retrieval is less sensitive to a single wrong character than literal matching. At generation, mark low-confidence pages so the answer can state that the source came from a scan and may contain recognition errors.
  6. Expected follow-up: can you auto-correct? Yes, but carefully. Dictionary or model based post-processing fixes some errors and breaks correct proper nouns. Restrict correction to low-confidence spans and keep the raw text so you can fall back.

  答题要点

  • 检索侧：形近字让字面匹配直接查不到，二元组分词还会让一个错字毁掉相邻两个词元，表现是召回悄悄下降且不报错。
  • 生成侧：模型能读懂大意，但关键实体出错时会自信地答错，引用校验也失去意义。
  • 入口层缓解：先判断有没有文本层再决定要不要 OCR，并保留原图链接供人工复核。
  • 检索层缓解：混合检索里的向量一路对个别错字不敏感，能兜住关键词一路的失手。
  • 生成层缓解：标出低置信度来源，让回答显式提示可能存在识别误差；自动纠错只对低置信片段做并保留原文。

  Key points

  • Retrieval: visually similar characters break literal matching, and bigram tokenisation lets one bad character corrupt two tokens, so recall drops silently.
  • Generation: the model reads through general noise but confidently repeats corrupted entities, and citation verification against a corrupted source proves nothing.
  • At ingest: check for a text layer before running OCR at all, and keep the source image for human verification.
  • At retrieval: hybrid search helps because dense retrieval tolerates a single wrong character better than literal matching.
  • At generation: flag low-confidence sources in the answer, and restrict auto-correction to low-confidence spans while keeping the raw text.

D9 混合检索与重排：两路召回、倒数排名融合，再用交叉编码器把前几名重新排一遍

• 你把纯向量检索换成了混合检索加重排，上线之后评估指标反而掉了。你会怎么排查？You replaced pure vector retrieval with hybrid search plus reranking, and after shipping it your eval metrics went down. How do you investigate?

  国内高频海外高频深入#hybrid-search#evaluation

  分析过程 · 先想清楚再作答

  1. 这题考的是你有没有真的做过分阶段归因。答「调一下权重再看看」就输了——那是在猜，不是在查。
  2. 第一步是拆档跑，不是改代码：纯关键词、纯向量、混合、混合加重排四档在**同一份评估集、同一个上下文预算**下各跑一遍。指标掉在哪一档就在哪一档找原因，这一步能立刻区分「融合坏了」和「重排坏了」。
  3. 第二步问一个具体问题：掉的是召回率还是排序指标？召回率掉说明答案根本没进上下文，是候选池或者预算的问题；排序指标掉而召回率没动，说明答案还在、只是被挤到了后面，那是融合权重或重排模型的问题。这两类故障的解法完全不同。
  4. 第三个常见根因是召回深度。每路取多少条这个旋钮方向反直觉：取深了不是更保险，是把噪声也一起投了票。我在一份 134 块的语料上实测过，每路从取 50 收到取 5，混合那一档的召回率从 87.5% 回到 93.8%、多跳档从 50% 回到 75%，而 nDCG 反而掉了近 0.1——两个指标会打架，先想清楚业务要哪个。
  5. 第四个根因是评估口径被悄悄改了。上下文预算、命中判定、候选池深度只要动过一个，新旧数字就不可比，这时候「掉了」可能根本不是真的掉了。
  6. 可预期的追问：怎么防止下次再踩？答把四档对照做成一条命令、把上一版报告存成基线、指标退步就以非 0 退出码拦住合并——这就是评估要先于优化的原因。

  How to reason about it · think before answering

  1. This question tests whether you have actually done stage-by-stage attribution. Answering `I would tune the weights and see` loses — that is guessing, not investigating.
  2. Step one is to run the stages apart, not to change code: pure keyword, pure vector, hybrid, and hybrid plus rerank, all on the **same eval set with the same context budget**. Whichever stage the drop appears in is where you look, and this alone separates `fusion is broken` from `reranking is broken`.
  3. Step two asks a specific question: did recall drop, or did the ranking metrics drop? A recall drop means the answer never entered the context at all — a candidate-pool or budget problem. Ranking metrics dropping while recall holds means the answer is still there but pushed down — a fusion-weight or rerank-model problem. The two failures have completely different fixes.
  4. A third common root cause is recall depth. This knob runs against intuition: going deeper is not safer, it lets noise vote too. On a 134-chunk corpus I measured that narrowing each route from 50 to 5 took hybrid recall from 87.5% back to 93.8% and multi-hop from 50% to 75%, while nDCG fell by almost 0.1. The metrics fight each other, so decide which one the product needs first.
  5. A fourth root cause is that the eval protocol quietly changed. Touch the context budget, the hit rule, or the candidate depth, and the old and new numbers stop being comparable — in which case the `drop` may not be a drop at all.
  6. Expected follow-up: how do you avoid this next time? Make the four-way comparison a single command, store the previous report as a baseline, and fail the build with a non-zero exit code on regression. That is precisely why evaluation comes before optimization.

  答题要点

  • 先拆档跑四种配置，在同一份评估集和同一个上下文预算下归因，不要一上来就调参。
  • 区分召回率掉与排序指标掉：前者是候选池或预算问题，后者是融合或重排问题。
  • 查召回深度：每路取太深会把噪声也投进融合，收窄反而可能救回召回率。
  • 确认评估口径没被改：预算、命中判定、候选池深度动过一个，新旧数字就不可比。
  • 把四档对照固化成一条命令加一份基线报告，指标退步直接拦住合并。

  Key points

  • Run all four configurations separately for attribution, on one eval set with one context budget, before touching any parameter.
  • Separate a recall drop from a ranking drop: the first is a candidate-pool or budget issue, the second is a fusion or rerank issue.
  • Check recall depth: taking too many per route lets noise vote, and narrowing it can bring recall back.
  • Confirm the eval protocol did not change; touching budget, hit rule, or candidate depth makes old and new numbers incomparable.
  • Freeze the four-way comparison into one command plus a baseline report, and block merges on regression.