面试题库

14 天 RAG：从检索到可信回答

D6 生成这一侧：上下文怎么排、引用怎么标、什么时候必须拒答，以及流式回答

• 怎么让模型的引用是真的而不是编的？说出一个不依赖模型自觉的方案。How do you make sure a model's citations are real rather than fabricated? Describe a scheme that does not rely on the model behaving well.

  国内高频海外高频进阶#citation-verification#grounding#hallucination

  分析过程 · 先想清楚再作答

  1. 题眼在「不依赖模型自觉」这半句。回答里只要出现「在提示词里强调请确保引用准确」，这题就答砸了——面试官问的正是提示词管不住的那部分。
  2. 先把问题拆成两半：引用要能验证，前提是它是一个**闭集里的符号**，不是一段自由文本。所以第一步是组装上下文时给每块材料一个编号，提示词里明确只能引用发出去的编号。让模型写「根据《某某手册》」是没法验证的，标题是它可以随口生成的字符串。
  3. 第二步是事后核对，两道闸缺一不可。第一道查编号存在性：发出去的是 1 到 5，出现 8 就一定是编的，一行代码判掉。第二道查实质重合：编号是真的、内容却对不上，这类更隐蔽，要算这句话的词元有多大比例能在被引块原文里找到，低于阈值判不通过。
  4. 算重合度时有个坑要主动说出来：先剔掉在多数块里都出现的高频词元，否则「文件」「系统」这种词会让随便哪一块都及格。这跟 BM25 用逆文档频率压常见词是同一个道理。
  5. 校验不过怎么办：把具体原因写成反馈打回去重生成一次，只给一次机会；连着两版都编说明材料本来就不支持，该走拒答而不是第三次重试。另外校验必须拿原文比对，不能拿压缩或改写过的材料比对，否则「校验通过」保证不了用户点开看到的东西。
  6. 可预期的追问：为什么不让模型自己再检查一遍？因为自检和生成是同一个模型的同一种倾向，它对自己编的东西没有独立信息源；而编号核对是一个确定性判断，成本几乎为零、结果可复现，这两点自检都做不到。

  How to reason about it · think before answering

  1. The phrase to catch is 'not relying on the model behaving well'. Any answer that boils down to 'tell the model to be accurate in the prompt' fails, because the prompt is exactly the part that cannot enforce this.
  2. Split the problem in two. Verifiability requires that a citation be a symbol from a closed set, not free text. So step one is numbering the blocks at assembly time and telling the model it may only cite the numbers it was given. 'According to the storage handbook' cannot be checked, because the title is a string the model can invent.
  3. Step two is post-hoc checking, with two gates. Gate one is existence: you handed out 1 through 5, so an 8 is fabricated, and that is a one-line check. Gate two is substantive overlap, which catches the sneakier case where the number is real but the block says something else. Measure what fraction of the sentence's terms appear in the cited block and reject below a threshold.
  4. Mention the trap in the overlap metric: drop terms that appear in most blocks first, otherwise generic words let any citation pass. It is the same reasoning behind inverse document frequency in BM25.
  5. On failure, feed the specific reason back and regenerate once, not repeatedly. Two fabricated drafts in a row means the material does not support the question, so refuse instead. Also verify against the original chunk text, never against a compressed or rewritten version, otherwise 'verified' says nothing about what the user sees.
  6. Expected follow-up: why not ask the model to self-check? Self-checking shares the generator's bias and has no independent source of truth, whereas number checking is deterministic, essentially free, and reproducible.

  答题要点

  • 引用必须是块编号这种闭集符号，不能是自由文本的文档标题——可验证性来自闭集，不来自措辞。
  • 两道闸：编号存在性，以及这句话与被引块原文的实质重合度，后者才拦得住「编号是真的、内容对不上」。
  • 算重合度前剔掉在多数块里都出现的高频词元，否则随便引哪一块都能及格。
  • 校验不过就带着具体原因打回重生成一次，只给一次机会，两版都编就转拒答。
  • 校验对象必须是用户能点开看到的原文，不是压缩或改写后的材料。

  Key points

  • Citations must be closed-set symbols such as block numbers, not free-text titles: verifiability comes from the closed set, not from wording.
  • Two gates: the number must exist, and the sentence must substantively overlap the cited block's original text, which is what catches real-number-wrong-content fabrication.
  • Strip terms that occur in most blocks before scoring overlap, or any citation will pass.
  • On failure, regenerate once with the concrete reason fed back; two bad drafts means refuse instead.
  • Always verify against the original text the user can open, never against a compressed or rewritten copy.
• 流式输出的场景下，你怎么保证吐出去的内容不会因为引用校验失败而需要撤回？In a streaming setup, how do you make sure nothing you have already sent needs to be retracted because its citation failed verification?

  国内高频海外高频深入#streaming#citation-verification#api-design

  分析过程 · 先想清楚再作答

  1. 这题在考一个真实的架构矛盾：流式要尽早出字，引用校验要等话说完才能核对。看回答里有没有出现「取舍」两个字，以及有没有把代价说清楚。
  2. 先说清矛盾在哪：一旦一个 token 发到了浏览器就撤不回来，你在末尾才发现第三句引用是编的，那句话已经在用户屏幕上了，只能补一句「刚才那句请忽略」，体验比不流式还糟。
  3. 给方案：按句缓冲。攒够一个完整句子就立刻校验一次，通过了才把这句连同已核实的引用发出去，没通过就整句丢掉。代价是首字延迟从一个 token 变成一句话，通常两三百毫秒，用户几乎察觉不到，而错误引用一旦上屏赔的是信任。
  4. 补两个实现细节，它们能证明你写过：流式模式没法用 JSON 输出（要等右花括号闭合才能解析），所以改成纯文本加行内标记，但校验必须和非流式共用同一套；标记要从正文里剥掉，正文保持干净，编号单独走校验再作为结构化数据发出去。
  5. 再补一条顺序上的讲究：生成前就能判的两条拒答线（分数过低、材料冲突）要在流开始之前发出去，用户不会先看到半句回答再被收回；生成后才能判的那条，在按句缓冲之下表现为一句都没发出来，收尾补一个拒答事件即可。
  6. 可预期的追问：那用户体验上的流式感是不是就没了？没有，句级流式在中文长回答里仍然是明显的渐进呈现；真要更细，可以在句子发出前先流一个「正在核对」的占位态，但不要流未校验的正文。

  How to reason about it · think before answering

  1. This tests a real architectural conflict: streaming wants the first token out early, citation verification cannot run until a statement is complete. Listen for whether the candidate names the trade-off and prices it.
  2. Name the conflict: once a token reaches the browser you cannot take it back. Discovering at the end that the third sentence cited a fabricated block leaves you posting 'please ignore that last sentence', which is worse than not streaming at all.
  3. Give the solution: buffer by sentence. As soon as a complete sentence lands, verify it, and only then emit it together with its verified citations; drop the whole sentence otherwise. The cost is that time-to-first-token becomes time-to-first-sentence, typically a few hundred milliseconds, which users barely notice, whereas a bad citation on screen costs trust.
  4. Add two implementation details that prove you have built it. Streaming cannot use JSON output because JSON is only parseable once closed, so switch to plain text with inline markers, while keeping exactly the same verifier as the non-streaming path. Strip the markers out of the prose and send the numbers as structured data after verification.
  5. Add the ordering point: the two rules decidable before generation, low score and source conflict, should be emitted before the stream starts, so the user never sees half an answer being withdrawn. The rule that needs generation shows up as 'no sentence was ever emitted', so close the stream with a refusal event.
  6. Expected follow-up: does this kill the streaming feel? No. Sentence-level streaming is still visibly progressive on long answers. If you need finer granularity, stream a 'checking sources' placeholder, but never stream unverified prose.

  答题要点

  • 矛盾在于发出去的内容撤不回来，而引用只有一句说完才能核对。
  • 解法是按句缓冲：攒够一句校验一次，通过才发，没通过整句丢掉。
  • 代价是首字延迟从一个 token 变成一句话，这个代价必须付也付得起。
  • 流式用不了 JSON，改纯文本加行内标记，但校验逻辑与非流式共用同一套；标记从正文剥出，编号作为结构化数据单独发。
  • 生成前能判的拒答要在流开始之前发出去，生成后才能判的那条以「一句都没发」的形式收尾补事件。

  Key points

  • The conflict: emitted text cannot be recalled, while a citation can only be checked once its sentence is complete.
  • The fix is sentence-level buffering: verify each completed sentence, emit only if it passes, drop the whole sentence if it does not.
  • The cost is time-to-first-sentence instead of time-to-first-token, which is affordable and worth paying.
  • Streaming cannot use JSON, so use inline markers in plain text while sharing one verifier with the non-streaming path; strip markers from the prose and send numbers as structured data.
  • Emit pre-generation refusals before the stream opens; the post-generation one manifests as an empty stream and is closed with a refusal event.