Interview Bank

How to reason about it · think before answering

1. The question is about where trust comes from. 'Citations are more convenient' is surface; the real difference is who guarantees the quote is real.
2. With prompting, both the quote and the page number are free text the model generates — it may paraphrase, it may misremember the page, and you cannot tell a real quote from an imagined one. With citations, the model emits citation intent in a standard format, the API parses and verifies it server-side, cited_text is guaranteed to exist in the document, and page_location comes from the API. Fidelity is enforced by the API rather than promised by the model.
3. Two side benefits: cited_text does not count toward output tokens, so it is cheaper than asking the model to copy; and the result is structured content blocks your UI can highlight and jump to without regex guessing.
4. When you can't: citations are incompatible with structured outputs (JSON Schema) — enabling both returns a 400. Either drop API-level citations and add a page field to the schema (one notch less reliable), or split into two calls: citations for facts, structured output for shaping.
5. Follow-ups: page semantics — start_page_number is 1-indexed and end_page_number is exclusive; document_index distinguishes sources. Can you audit citation quality? Yes — string-match cited_text against the source, or sample manually.

分析过程 · 先想清楚再作答

1. 这题考的是「可信度从哪来」。答成「citations 更方便」是表面；本质区别是谁来保证引用的真实性。
2. 拆法：提示词方案里，引用和页码都是模型生成的自由文本——它可能顺手改写原文、可能记错页码，你无法区分「真引用」和「自以为引用」。citations 方案里，模型内部以标准格式输出引用意图，API 在服务端解析并核对，返回的 cited_text 一定是文档里真实存在的段落，page_location 的页码由 API 给出。真实性由 API 保证而不是由模型自觉保证。
3. 附带的两点好处：cited_text 不计入输出 token，比让模型抄原文便宜；返回是结构化的内容块，程序可以直接高亮、跳转，不用正则去猜「第 3 页」出现在哪。
4. 不能用的场景：与结构化输出（JSON Schema）不兼容，二者同开会报 400；此时要么放弃 API 级引用、在 schema 里留 page 字段让模型自己填（可靠性差一档），要么分两步：先 citations 拿事实，再用结构化输出整理。
5. 可预期的追问：页码字段的语义？start_page_number 从 1 开始，end_page_number 不包含；多文档时 document_index 区分来源。再追问「能否验证引用质量」——能，用 cited_text 与原文做字符串比对，或抽样人工核对。

How to reason about it · think before answering

1. The phrase to catch is 'not relying on the model behaving well'. Any answer that boils down to 'tell the model to be accurate in the prompt' fails, because the prompt is exactly the part that cannot enforce this.
2. Split the problem in two. Verifiability requires that a citation be a symbol from a closed set, not free text. So step one is numbering the blocks at assembly time and telling the model it may only cite the numbers it was given. 'According to the storage handbook' cannot be checked, because the title is a string the model can invent.
3. Step two is post-hoc checking, with two gates. Gate one is existence: you handed out 1 through 5, so an 8 is fabricated, and that is a one-line check. Gate two is substantive overlap, which catches the sneakier case where the number is real but the block says something else. Measure what fraction of the sentence's terms appear in the cited block and reject below a threshold.
4. Mention the trap in the overlap metric: drop terms that appear in most blocks first, otherwise generic words let any citation pass. It is the same reasoning behind inverse document frequency in BM25.
5. On failure, feed the specific reason back and regenerate once, not repeatedly. Two fabricated drafts in a row means the material does not support the question, so refuse instead. Also verify against the original chunk text, never against a compressed or rewritten version, otherwise 'verified' says nothing about what the user sees.
6. Expected follow-up: why not ask the model to self-check? Self-checking shares the generator's bias and has no independent source of truth, whereas number checking is deterministic, essentially free, and reproducible.

分析过程 · 先想清楚再作答

1. 题眼在「不依赖模型自觉」这半句。回答里只要出现「在提示词里强调请确保引用准确」，这题就答砸了——面试官问的正是提示词管不住的那部分。
2. 先把问题拆成两半：引用要能验证，前提是它是一个**闭集里的符号**，不是一段自由文本。所以第一步是组装上下文时给每块材料一个编号，提示词里明确只能引用发出去的编号。让模型写「根据《某某手册》」是没法验证的，标题是它可以随口生成的字符串。
3. 第二步是事后核对，两道闸缺一不可。第一道查编号存在性：发出去的是 1 到 5，出现 8 就一定是编的，一行代码判掉。第二道查实质重合：编号是真的、内容却对不上，这类更隐蔽，要算这句话的词元有多大比例能在被引块原文里找到，低于阈值判不通过。
4. 算重合度时有个坑要主动说出来：先剔掉在多数块里都出现的高频词元，否则「文件」「系统」这种词会让随便哪一块都及格。这跟 BM25 用逆文档频率压常见词是同一个道理。
5. 校验不过怎么办：把具体原因写成反馈打回去重生成一次，只给一次机会；连着两版都编说明材料本来就不支持，该走拒答而不是第三次重试。另外校验必须拿原文比对，不能拿压缩或改写过的材料比对，否则「校验通过」保证不了用户点开看到的东西。
6. 可预期的追问：为什么不让模型自己再检查一遍？因为自检和生成是同一个模型的同一种倾向，它对自己编的东西没有独立信息源；而编号核对是一个确定性判断，成本几乎为零、结果可复现，这两点自检都做不到。