Interview Bank

RAG in 14 Days: From Retrieval to Trustworthy Answers

D1 Why Retrieve at All: Hallucination, Knowledge Cutoffs, and the Cost of Long Context; a Minimal Keyword-Only RAG

• Context windows are now in the millions of tokens. Does that make the retrieval step obsolete?上下文窗口已经做到上百万 token 了，检索这一步会被淘汰吗？

  Common in ChinaCommon overseasDeep dive#long-context#cost#system-design

  How to reason about it · think before answering

  1. This is a position question and it is easy to answer as a binary. The signal is whether you separate what fits technically from what is worth paying for on every request.
  2. Concede the valid half first: bigger windows genuinely absorb part of the use case. For an internal tool over a few dozen stable documents with low traffic, stuffing everything in is the right call and building a retrieval stack would be over-engineering.
  3. Then give three reasons it does not absorb the rest. Cost is the first: context is billed per request, so the same corpus is paid for on every one of ten thousand queries, whereas retrieval only pays for the passages it returns. Prompt caching softens this but does not remove it.
  4. Scale is the second: enterprise corpora run to hundreds of thousands of documents and no window holds them. Attribution and access control are the third: pointing an answer at a specific passage, and showing each user only what they are permitted to see, both have to happen before the material reaches the model.
  5. Add the empirical point: as the supplied material grows, models become less reliable at locating the one relevant fact inside it. More context is not automatically better; fewer and more precise passages often win.
  6. Expected follow-up: does retrieval change shape? Yes. Larger windows allow bigger chunks and more of them, which relieves pressure on reranking and compression. Retrieval gets coarser, it does not disappear.

  分析过程 · 先想清楚再作答

  1. 这是一道立场题，容易答成非黑即白。判断你有没有做过的地方在于：会不会区分「技术上能不能塞进去」和「工程上该不该每次都塞」，只谈前者的答案一听就是纸上谈兵。
  2. 先承认对方有道理的部分：窗口变大确实吃掉了检索的一部分场景。几十篇文档、更新不频繁、调用量不大的内部工具，直接全塞是最省事的选择，为它建一套检索系统是过度设计。
  3. 再给三条它吃不掉的理由。第一是成本：材料是按次计费的，同一份材料被问一万次就要付一万次，而检索只付取回的那几段；预填充缓存能缓解但不能消除，缓存也有有效期和命中率。
  4. 第二是规模：企业知识库动辄几十万篇，再大的窗口也塞不下，检索是唯一的入口。第三是归因与权限：答案要指回具体某一段，以及不同的人只能看到自己有权访问的材料——这两件事必须在把材料喂给模型之前完成，窗口再大也不解决。
  5. 还要补一条经验事实：材料变多之后，模型在长上下文里定位关键信息的稳定性会下降，出现「读了但没读到」。所以「全塞」并不总是等于「效果更好」，很多时候少而准反而更好。
  6. 可预期的追问：那检索的形态会不会变？会——窗口变大之后，取回的块可以更大、条数可以更多，重排与压缩的压力变小，检索从「精挑几句」变成「粗筛一批」。趋势是检索的粒度变粗，不是检索消失。

  Key points

  • Separate whether it fits from whether it is worth paying for on every request.
  • Small, stable, low-traffic corpora can legitimately be stuffed whole; building retrieval for them is over-engineering.
  • Three reasons retrieval survives: per-request cost, corpora too large for any window, and attribution plus access control that must happen before the model sees the material.
  • More supplied context reduces the reliability of locating a single fact, so stuffing everything is not automatically better.
  • The trend is coarser retrieval — bigger chunks, more of them, less reranking pressure — not the removal of retrieval.

  答题要点

  • 先区分「能不能塞进去」和「该不该每次都塞」，前者是技术问题，后者是成本问题。
  • 小规模、低频、少变的语料确实可以直接全塞，为它建检索系统是过度设计。
  • 检索不会被淘汰的三个理由：按次计费的成本、几十万篇塞不下的规模、必须在喂给模型之前完成的归因与权限过滤。
  • 材料越多，模型定位关键信息的稳定性越差，全塞不等于效果更好。
  • 趋势是检索粒度变粗——块更大、条数更多、重排压力变小，而不是检索消失。

D3 Getting Documents In: Parsing PDF and HTML, Tables and Scans, Cleaning Rules, and Metadata You Must Keep

• OCR output from scanned documents carries a non-trivial error rate. How does that noise propagate into retrieval and generation, and how do you mitigate it?扫描件走光学字符识别之后错字率不低，这些噪声会怎样影响检索和生成？怎么缓解？

  Common in ChinaCommon overseasDeep dive#ocr#data-quality#hybrid-search

  How to reason about it · think before answering

  1. This tests whether you can trace propagation rather than recite that OCR makes mistakes. The differentiator is separating how retrieval fails from how generation fails, because the two failure modes are entirely different.
  2. Retrieval first. Chinese OCR errors are mostly visually similar characters. Keyword search is literal, so one wrong character makes the term unmatchable, and bigram tokenisation makes it worse because a single wrong character corrupts two adjacent tokens. Recall drops quietly and nothing raises an error.
  3. Generation second. The model usually reads through minor noise, but when the corrupted token is a key entity such as a name, a model number, an amount or a date, it answers confidently with the wrong value. Citation checking degrades too: verifying against a source that is itself wrong proves nothing.
  4. Mitigate in three layers. At ingest, use an empty-text assertion to decide whether the PDF even needs OCR, and keep a link to the original image so a human can verify.
  5. At retrieval, hybrid search absorbs some of the damage because dense retrieval is less sensitive to a single wrong character than literal matching. At generation, mark low-confidence pages so the answer can state that the source came from a scan and may contain recognition errors.
  6. Expected follow-up: can you auto-correct? Yes, but carefully. Dictionary or model based post-processing fixes some errors and breaks correct proper nouns. Restrict correction to low-confidence spans and keep the raw text so you can fall back.

  分析过程 · 先想清楚再作答

  1. 这题考的是你会不会顺着链条推传导，而不是背「OCR 会有错字」这句废话。判据是有没有分别说清「检索侧怎么错」和「生成侧怎么错」——它们的失效方式完全不同。
  2. 先说检索侧。中文 OCR 的错主要是形近字，「已」认成「己」、「板」认成「版」。关键词检索是字面匹配，一个字错了这个词就查不到；更隐蔽的是二元组分词会连带毁掉相邻两个词元，一个错字影响的其实是两处。这一路的表现是召回悄悄掉下去，而且不报错。
  3. 再说生成侧。错字进了上下文，模型往往能读懂大意，但一旦是关键实体（人名、型号、金额、日期）出错，它会照着错的答，而且答得很自信。更麻烦的是引用校验也会跟着失效——原文本身就是错的，校验通过了也没意义。
  4. 缓解按三层说。入口层：先用空文本比例这类断言判断这份 PDF 有没有文本层，有就别走 OCR；真要走，保留原图链接以便人工复核。
  5. 检索层：靠混合检索兜底，向量一路对个别错字不敏感，能补上关键词一路的失手，这是 D9 那套东西在这里的具体价值。生成层：把低置信度的页面标出来，让模型在引用它们时明确提示「该材料来自扫描件，可能有识别误差」。
  6. 可预期的追问：能不能自动纠错？可以但要克制——用词典或模型做后处理会修好一批，也会「修」坏一批原本正确的专有名词。稳妥的做法是只对置信度低的片段做纠错，并且保留原文以便回退。

  Key points

  • Retrieval: visually similar characters break literal matching, and bigram tokenisation lets one bad character corrupt two tokens, so recall drops silently.
  • Generation: the model reads through general noise but confidently repeats corrupted entities, and citation verification against a corrupted source proves nothing.
  • At ingest: check for a text layer before running OCR at all, and keep the source image for human verification.
  • At retrieval: hybrid search helps because dense retrieval tolerates a single wrong character better than literal matching.
  • At generation: flag low-confidence sources in the answer, and restrict auto-correction to low-confidence spans while keeping the raw text.

  答题要点

  • 检索侧：形近字让字面匹配直接查不到，二元组分词还会让一个错字毁掉相邻两个词元，表现是召回悄悄下降且不报错。
  • 生成侧：模型能读懂大意，但关键实体出错时会自信地答错，引用校验也失去意义。
  • 入口层缓解：先判断有没有文本层再决定要不要 OCR，并保留原图链接供人工复核。
  • 检索层缓解：混合检索里的向量一路对个别错字不敏感，能兜住关键词一路的失手。
  • 生成层缓解：标出低置信度来源，让回答显式提示可能存在识别误差；自动纠错只对低置信片段做并保留原文。

D4 Chunking Strategies: Five Approaches — Fixed, Recursive, Structure-Based, Parent-Child, and Semantic — and Choosing by Evaluation, Not Intuition

• Semantic chunking costs considerably more than recursive splitting. How would you prove to your team that the money is well spent?语义切分比递归切分贵不少，你怎么向团队证明这笔钱值得花？

  Common in ChinaCommon overseasDeep dive#chunking#evaluation#cost

  How to reason about it · think before answering

  1. This looks like a technical question but it tests whether you can run a controlled technical argument. Launching into how semantic chunking works answers a different question.
  2. Step one is to concede that it may well not be worth it. The gain comes from documents that have no usable structure; if your knowledge base is well-formed documents, the authors' heading hierarchy already did the semantic split for free and the money is likely wasted.
  3. Step two is translating 'worth it' into three measurable numbers: how much the metric moved (hit rate on the same golden set under the same token budget), how much latency moved (chunking is offline, but the end-to-end update path changes), and how much it costs (the initial full embedding pass plus recomputation amortised over update frequency).
  4. Step three is the control. Recursive splitting is the baseline, semantic chunking the treatment, and they must share the corpus, the questions, the context budget and the retriever. Change one variable only; a two-variable experiment proves nothing.
  5. Step four is a decision threshold rather than an impression. For example: below three points of hit-rate gain, no; above five points with recomputation inside the monthly budget, yes; in between, roll it out on one document class first. Fix the threshold before you run the numbers, or you will quietly bend it to fit them.
  6. Expect the follow-up: is there a cheaper way to the same gain. Yes — try structural splitting first, since it is free and often nearly as good, and if the structure really is unusable, apply semantic chunking only to the high-value subset rather than the whole corpus.

  分析过程 · 先想清楚再作答

  1. 这题表面问技术，实际考的是你会不会做一次带对照组的技术论证。上来就讲语义切分原理的人，答的是另一道题。
  2. 第一步是先承认它可能不值。语义切分的收益来自「文档没有可用的结构」；如果知识库是结构良好的文档，作者的标题层级已经免费替你做完了语义切分，这时候花的钱大概率打水漂。**先说清适用前提，再谈证明，这一步就把大多数候选人区分开了。**
  3. 第二步是把「值不值」翻译成可测的三笔账：指标涨了多少（同一批标准问题、同一个 token 预算下的命中率）、延迟涨了多少（切块是离线的，但更新链路的端到端时间会变）、钱涨了多少（首次全量 embedding 的费用，加上按更新频率折算的重算费用）。只报第一笔的论证不成立。
  4. 第三步是设计对照。递归切分是基线，语义切分是实验组，两组必须用同一份语料、同一批问题、同一个上下文预算、同一个检索器，只改切法这一个变量。改两个变量的实验，结论一文不值。
  5. 第四步是给决策一个门槛，而不是给一个感想。比如：命中率相对基线提升低于三个百分点就不上；提升超过五个百分点且重算成本在月度预算内就上；中间地带先在一类文档上灰度。**门槛要在跑数字之前定好**，否则你会不自觉地去迁就已经跑出来的结果。
  6. 可预期的追问是「有没有更便宜的办法拿到同样的收益」。答有：先试按结构切，它零成本且效果常常接近；结构确实不可用时，再考虑只对高价值的那一部分文档做语义切分，而不是全量上。

  Key points

  • Start with the precondition: the gain comes from documents without usable structure, so on well-formed documents it usually is not worth it.
  • Translate 'worth it' into three numbers — hit rate, latency, and cost. Reporting only the first is not an argument.
  • Run a controlled comparison: same corpus, same golden set, same context budget, same retriever, with the splitting strategy as the only variable.
  • Fix the decision threshold before running the numbers so you cannot bend it to fit the result afterwards.
  • Try free structural splitting first, and if semantic chunking is genuinely needed, apply it to the high-value subset rather than the entire corpus.

  答题要点

  • 先讲适用前提：语义切分的收益来自文档没有可用结构，结构良好的文档上它大概率不值。
  • 把「值不值」翻译成三笔账：命中率涨多少、延迟涨多少、钱涨多少，只报第一笔不算论证。
  • 做对照实验：同语料、同问题集、同上下文预算、同检索器，只改切法一个变量。
  • 决策门槛必须在跑数字之前定好，避免事后迁就结果。
  • 先试零成本的按结构切；确需语义切分时也优先只覆盖高价值文档，而不是全量上。

D5 Vector Indexes and Store Selection: HNSW vs. Inverted File, Quantization to Save Memory, Filtered Queries and Multi-Tenant Isolation

• Why does a vector search with a WHERE clause return fewer results than expected, and what are the fixes and their costs?为什么加了 WHERE 条件的向量检索会漏结果？有哪几种修法，代价分别是什么？

  Common in ChinaCommon overseasDeep dive#filtering#iterative-scan#recall

  How to reason about it · think before answering

  1. This is the question that separates people who ran a demo from people who ran this in production. The tell is whether you distinguish missing rows from mis-ordered rows.
  2. State the mechanism in one sentence: with approximate indexes, filtering is applied after the index scan. The index first collects ef_search candidates by distance, and only then applies the WHERE clause to that batch.
  3. Do the arithmetic out loud: a condition matching 1% of rows against a default candidate list of 40 leaves well under one row on average. That is why the query looks broken even though the rows exist.
  4. Split the failure into two kinds. Too few rows returned is one; enough rows but the wrong ones ranked first is the other. They have different fixes, and conflating them signals inexperience.
  5. Fix one is iterative scanning, available since pgvector 0.8.0: when too many candidates are filtered out, keep scanning more of the index until enough results are found. Strict ordering keeps exact distance order, relaxed ordering trades slight reordering for better recall, and both cost latency.
  6. Fix two is making the filter apply first: a plain index on the filter column for highly selective conditions, a partial index when there are only a few distinct values, list partitioning when there are many. The costs are losing the approximate speedup, index count exploding per value, and DDL plus operational complexity.
  7. Expected follow-up: how do you pick? Check the returned row count first. Too few means iterative scanning; enough rows with low recall means raising probes or ef_search, or switching to pre-filtering.

  分析过程 · 先想清楚再作答

  1. 这题是本天的核心，也是最能筛掉「只跑过 demo」的人的一题。题眼在「漏」这个字：能不能说清楚漏的是条数还是排序，直接决定你被归到哪一档。
  2. 先讲机制，一句话就够：近似索引的过滤发生在索引扫描之后。索引先按距离取回 ef_search 个候选，然后才拿 WHERE 去筛这一批。条件命中率越低，活下来的越少——命中 1% 的条件配默认的 40 个候选，平均只剩零点几条。
  3. 然后把漏召回拆成两类，这是拿分点：一类是**结果条数不够**，十条只给了一两条；另一类是**条数够但排序不对**，十条都在只是排错了。两类的修法完全不同，混为一谈说明没真跑过。
  4. 修法一是迭代扫描（pgvector 0.8.0 起）：候选被过滤掉太多时自动回索引里继续扫，直到凑够。它只解决第一类。两种模式的取舍要说清楚——严格顺序保证结果按距离排好，宽松顺序允许略微乱序换更高召回，代价都是延迟明显上升。
  5. 修法二是预过滤，即让过滤条件先生效：条件很挑剔时给过滤列建普通索引走精确检索，取值只有少数几个时建部分索引，取值很多时按值做列表分区。代价分别是失去近似索引的加速、索引数量随取值爆炸、以及 DDL 与运维复杂度上升。
  6. 可预期的追问：怎么判断该用哪一种？给一条可执行的判据——先看返回条数够不够。不够是第一类，先试迭代扫描；够了但召回低是第二类，只能加大 probes 或 ef_search，或者干脆改成预过滤。

  Key points

  • With approximate indexes the filter runs after the index scan, so a selective condition wipes out most candidates and the query returns too few rows.
  • There are two failure modes: too few rows, and enough rows in the wrong order. Always check the returned count first.
  • Iterative scanning fixes only the first. Strict ordering preserves distance order, relaxed ordering gives better recall, and both raise latency noticeably.
  • Pre-filtering is the alternative: index the filter column for exact search, use a partial index for a few distinct values, partition by value for many. Costs are losing the approximate speedup, index sprawl, and operational complexity.
  • The second failure mode is only fixed by raising probes or ef_search; iterative scanning does nothing for it.

  答题要点

  • 近似索引的过滤发生在索引扫描之后，条件命中率低时候选几乎被筛光，所以返回条数不够。
  • 漏召回分两类：条数不够，和条数够但排序不对。判断顺序永远是先看返回条数。
  • 迭代扫描只修第一类，严格顺序保序、宽松顺序召回更高，代价是延迟明显上升。
  • 预过滤是另一条路：过滤列建索引走精确检索、取值少建部分索引、取值多按值分区，代价依次是失去索引加速、索引数量爆炸、运维复杂度上升。
  • 第二类只能靠加大 probes 或 ef_search，迭代扫描对它完全无效。

D6 The Generation Side: Ordering Context, Labeling Citations, When You Must Refuse to Answer, and Streaming Responses

• How do you set the refusal threshold for a knowledge-base assistant, and what does it cost you when the threshold is too high or too low?知识库问答的拒答阈值怎么定？定高了和定低了各自的代价是什么？

  Common in ChinaCommon overseasDeep dive#refusal#thresholds#evaluation

  How to reason about it · think before answering

  1. What is really being tested: do you know that refusal is several rules rather than one threshold, and do you set thresholds from data. An answer that mentions only a score cutoff shows you have only touched the surface.
  2. Break refusal into three rules with different timing. Score too low: decidable before generation, saving a model call. Sources conflict: also decidable before generation, by finding differing numbers about the same thing across blocks. You then either present both with their update dates, or pick the newer one when an authoritative signal backs it, such as meeting notes that flagged the discrepancy. Which of the two is a product decision, but silently letting the model pick is never an option. Question outside coverage: only decidable after generation, when citation verification leaves you with zero verified citations.
  3. Stress that the three responses must read differently. 'Nothing relevant in the knowledge base, try rephrasing or check whether the document was ingested' is a different instruction to the user than 'we found related documents but none of them answers this'. Collapsing both into 'sorry, I don't know' throws away information.
  4. Then the cost half. Too high: answerable questions get blocked, the user is told nothing was found while the material is in fact indexed. That is the most trust-damaging failure and it is nearly invisible in logs. Too low: weak passages enter the context and the model answers from irrelevant material, which is worse because the answer still looks cited.
  5. How to set it: run a set of questions with known answers and known non-answers, look at where the two score distributions separate, and pick a point according to which error you fear more. Scores have no absolute scale, so the deliverable is the procedure, not the number.
  6. Expected follow-up: what if one score threshold is not enough? Add signals rather than tuning the number: the gap between top and second score, the number of hits above threshold, and the post-generation verification result are all steadier than the raw score.

  分析过程 · 先想清楚再作答

  1. 这题真正在考的是：你有没有意识到拒答不是一个阈值，而是好几条判据；以及你定阈值靠不靠数据。只谈一个分数阈值的回答，说明只做过最浅的一层。
  2. 先把拒答拆成三条线，它们的触发时机完全不同。检索分数太低：生成之前就能判，省一次模型调用。材料互相矛盾：也在生成之前判，代码在块之间找同一件事的不同数字，检出后要么并列两种说法与各自的更新日期，要么在有权威信号（比如一份点破了这条不一致的会议纪要）时按更新日期择一——选哪条是产品决策，但无论如何不能让模型自己悄悄挑一个。问题超出材料覆盖范围：只能在生成之后判，判据是跑完引用校验一条有效引用都没有。
  3. 强调三种话术必须不同。第一种要说「库里没有相关材料，换个说法或确认资料是否入库」，第三种要说「找到了相关文档但里面没有能直接回答的内容」——用户的下一步动作完全不同，混成一句「抱歉我不知道」等于把信息扔了。
  4. 再答代价这一半。定高了：能答的问题被挡在门外，用户看到查不到而材料其实在库里，这是最伤信任的一种错，而且它在日志里几乎不可见。定低了：低分噪声材料进上下文，模型拿着不相关的东西硬答，错误反而更隐蔽，因为回答看起来还带着引用。
  5. 怎么定：拿一批已知有答案和已知没答案的问题跑一遍，看两组的分数分布在哪里分开，按你更怕哪种错来取点。分数是没有绝对量纲的，换语料、换检索方式都要重定，所以真正要交付的是这套定阈值的流程，不是那个数字。
  6. 可预期的追问：单一分数阈值不够怎么办？答案是加判据而不是调数字——最高分与次高分的差、命中块数、以及生成后的引用校验结果，都是比原始分数更稳的信号。

  Key points

  • Refusal is three rules, not one: low score and source conflict decided before generation, out-of-coverage decided after generation from the verification result.
  • On conflict, presenting both versions versus picking the newer one is a product decision; picking only holds up when an authoritative signal backs it.
  • The three responses must be worded differently because each implies a different next action for the user.
  • Too high blocks answerable questions; the user is told nothing exists while it does, which is the most damaging and least visible failure.
  • Too low lets weak passages in, producing errors that are harder to spot because the answer still carries citations.
  • Set it by comparing score distributions over answerable and unanswerable question sets, then choose based on which error is worse; re-tune whenever the corpus or retriever changes.

  答题要点

  • 拒答不是一条线而是三条：分数过低、材料冲突（都在生成前判）、超出材料覆盖范围（只能生成后按引用校验结果判）。
  • 冲突检出后并列两说还是按更新日期择一，是产品决策；只有在有权威信号背书时择一才站得住，否则老实并列。
  • 三种情况的话术必须不同，因为它们给用户的下一步动作不同。
  • 定高了会把能答的问题挡住，用户看到查不到而材料其实在库里，最伤信任且日志里看不见。
  • 定低了会让噪声材料进上下文，错误更隐蔽，因为回答看起来仍然带着引用。
  • 定法是拿已知有答案与已知没答案的两组问题跑分数分布，按更怕哪种错取点；换语料或换检索方式都要重定。
• In a streaming setup, how do you make sure nothing you have already sent needs to be retracted because its citation failed verification?流式输出的场景下，你怎么保证吐出去的内容不会因为引用校验失败而需要撤回？

  Common in ChinaCommon overseasDeep dive#streaming#citation-verification#api-design

  How to reason about it · think before answering

  1. This tests a real architectural conflict: streaming wants the first token out early, citation verification cannot run until a statement is complete. Listen for whether the candidate names the trade-off and prices it.
  2. Name the conflict: once a token reaches the browser you cannot take it back. Discovering at the end that the third sentence cited a fabricated block leaves you posting 'please ignore that last sentence', which is worse than not streaming at all.
  3. Give the solution: buffer by sentence. As soon as a complete sentence lands, verify it, and only then emit it together with its verified citations; drop the whole sentence otherwise. The cost is that time-to-first-token becomes time-to-first-sentence, typically a few hundred milliseconds, which users barely notice, whereas a bad citation on screen costs trust.
  4. Add two implementation details that prove you have built it. Streaming cannot use JSON output because JSON is only parseable once closed, so switch to plain text with inline markers, while keeping exactly the same verifier as the non-streaming path. Strip the markers out of the prose and send the numbers as structured data after verification.
  5. Add the ordering point: the two rules decidable before generation, low score and source conflict, should be emitted before the stream starts, so the user never sees half an answer being withdrawn. The rule that needs generation shows up as 'no sentence was ever emitted', so close the stream with a refusal event.
  6. Expected follow-up: does this kill the streaming feel? No. Sentence-level streaming is still visibly progressive on long answers. If you need finer granularity, stream a 'checking sources' placeholder, but never stream unverified prose.

  分析过程 · 先想清楚再作答

  1. 这题在考一个真实的架构矛盾：流式要尽早出字，引用校验要等话说完才能核对。看回答里有没有出现「取舍」两个字，以及有没有把代价说清楚。
  2. 先说清矛盾在哪：一旦一个 token 发到了浏览器就撤不回来，你在末尾才发现第三句引用是编的，那句话已经在用户屏幕上了，只能补一句「刚才那句请忽略」，体验比不流式还糟。
  3. 给方案：按句缓冲。攒够一个完整句子就立刻校验一次，通过了才把这句连同已核实的引用发出去，没通过就整句丢掉。代价是首字延迟从一个 token 变成一句话，通常两三百毫秒，用户几乎察觉不到，而错误引用一旦上屏赔的是信任。
  4. 补两个实现细节，它们能证明你写过：流式模式没法用 JSON 输出（要等右花括号闭合才能解析），所以改成纯文本加行内标记，但校验必须和非流式共用同一套；标记要从正文里剥掉，正文保持干净，编号单独走校验再作为结构化数据发出去。
  5. 再补一条顺序上的讲究：生成前就能判的两条拒答线（分数过低、材料冲突）要在流开始之前发出去，用户不会先看到半句回答再被收回；生成后才能判的那条，在按句缓冲之下表现为一句都没发出来，收尾补一个拒答事件即可。
  6. 可预期的追问：那用户体验上的流式感是不是就没了？没有，句级流式在中文长回答里仍然是明显的渐进呈现；真要更细，可以在句子发出前先流一个「正在核对」的占位态，但不要流未校验的正文。

  Key points

  • The conflict: emitted text cannot be recalled, while a citation can only be checked once its sentence is complete.
  • The fix is sentence-level buffering: verify each completed sentence, emit only if it passes, drop the whole sentence if it does not.
  • The cost is time-to-first-sentence instead of time-to-first-token, which is affordable and worth paying.
  • Streaming cannot use JSON, so use inline markers in plain text while sharing one verifier with the non-streaming path; strip markers from the prose and send numbers as structured data.
  • Emit pre-generation refusals before the stream opens; the post-generation one manifests as an empty stream and is closed with a refusal event.

  答题要点

  • 矛盾在于发出去的内容撤不回来，而引用只有一句说完才能核对。
  • 解法是按句缓冲：攒够一句校验一次，通过才发，没通过整句丢掉。
  • 代价是首字延迟从一个 token 变成一句话，这个代价必须付也付得起。
  • 流式用不了 JSON，改纯文本加行内标记，但校验逻辑与非流式共用同一套；标记从正文剥出，编号作为结构化数据单独发。
  • 生成前能判的拒答要在流开始之前发出去，生成后才能判的那条以「一句都没发」的形式收尾补事件。

D7 Week One Capstone: Assembling Six Days of Parts Into a One-Command Question-Answering Service, and a Retrospective

• What is the biggest risk in the RAG service you just assembled, and how would you prove that judgment?你刚拼出来的这个检索问答系统，现在最大的风险在哪里？你打算怎么证明这个判断？

  Common in ChinaCommon overseasDeep dive#evaluation#risk-assessment#retrospective

  How to reason about it · think before answering

  1. There are two halves here and the second is the real question. Naming a risk is easy; giving a method that could falsify your own claim is what separates answers from opinions.
  2. Rule out two common wrong answers: 'hallucination' is too vague to act on, and 'latency' mistakes a visible problem for the biggest one.
  3. The biggest risk is the absence of evaluation. Chunk size, top-k, thresholds and route weights were all guessed, and that makes every other risk unverifiable: you cannot even say whether a change helped.
  4. How to prove it: build a question set from the corpus with known answer documents, deliberately including unanswerable and multi-hop questions; implement recall and ranking metrics; produce a baseline for the current configuration; then move one parameter back and forth and watch whether the metrics move. If they do not move at all, the evaluation set is wrong, not the system.
  5. Add the accounting rule: every optimisation reports three numbers, metric gain, latency added and cost added. A claim with only the first is not usable.
  6. Expected follow-up: how large must the set be? Start with roughly twenty questions covering the main question types to catch obvious regressions, then grow toward the real distribution once you have actual user questions. Chasing size first only yields questions you invented yourself.

  分析过程 · 先想清楚再作答

  1. 这题有两半，后半句才是题眼。说出一个风险不难，难的是给出一个能证伪你自己判断的方法——答不出后半句，前半句就只是意见。
  2. 先排除两个常见的错误答案：说「幻觉」太笼统，没有指向任何可动的地方；说「延迟」则是把看得见的问题当成最大风险。
  3. 真正的最大风险是**没有评估**：切块大小、取几条、门槛定多少、两路怎么加权，全是拍出来的。它最重要的地方在于它让所有其他风险都无法验收——你连「改了之后变好还是变坏」都说不出口。
  4. 怎么证明：先从语料反向出一份带标准答案文档的问题集，刻意掺进无答案问题和需要跨文档的多跳问题；再实现召回率与排序指标，给当前配置跑出一个基线；然后把一个参数来回改两次，看指标动不动。如果指标对参数完全不敏感，说明是评估集有问题，不是系统没问题。
  5. 补一句成本口径：每一项优化都要同时报三笔账——指标涨了多少、延迟涨了多少、钱涨了多少。只报第一笔的结论不能用。
  6. 可预期的追问：评估集多大才够？先做二十题能覆盖主要问题类型的小集，用它挡住明显的退步；等真实用户问题攒起来，再按真实分布扩到几百题。一上来就追求规模，只会得到一堆自己出的、跟真实用法无关的题。

  Key points

  • The biggest risk is having no evaluation: every parameter was guessed, so no change can be judged.
  • Prove it by building a golden set with known answer documents, including unanswerable and multi-hop questions, then baseline the current configuration.
  • Validate the set itself by perturbing parameters: metrics that never move mean the questions are wrong.
  • Report three numbers per optimisation: metric gain, added latency, added cost.
  • Start small but well covered, then grow toward the real question distribution.

  答题要点

  • 最大的风险是没有评估：所有参数都是拍的，导致任何改动的好坏都无法判断。
  • 证明方式是先建标准答案集，刻意包含无答案问题与多跳问题，再跑出当前配置的基线。
  • 用参数扰动反过来验证评估集本身：指标对参数完全不敏感，说明题出得有问题。
  • 每项优化同时报三笔账：指标、延迟、成本；只报指标的结论不能用。
  • 评估集先小而全，覆盖问题类型即可，等真实问题攒起来再按真实分布扩大。