面试题库

14 天 RAG：从检索到可信回答

D11 高级索引：父子文档、摘要索引、上下文检索，以及树状聚合与图检索的取舍

• 上下文检索要给每个块调一次模型，这笔一次性成本怎么估？有哪些办法能压下来？Contextual retrieval needs one model call per chunk. How do you estimate that one-off cost, and what levers bring it down?

  国内高频海外高频深入#contextual-retrieval#prompt-caching#cost

  分析过程 · 先想清楚再作答

  1. 这题考的是你有没有真的算过账。只会说『用提示词缓存就便宜了』属于听过没做过——面试官会追问缓存到底省在哪一项上。
  2. 先把成本拆开：一次性 = 每块的输入 + 输出 + 全量 embedding；每次查询 = 块头在重排和上下文里各被读一遍。**这两笔要分开记**，因为它们随业务量的增长方式完全不同。
  3. 一次性那笔的主项是『同一篇文档被重复读了多少遍』。一篇切成 n 块就要读 n 遍，这是成本的大头。提示词缓存省的正是这一项：把整篇放在提示词最前面并标记为可缓存，第一块付一次缓存写入，后面 n-1 块只付缓存读取，而读取价通常比输入价低一个数量级。
  4. 顺序不能反：缓存按前缀匹配，整篇必须在前、块内容在后。把变化的块放前面，前缀次次都变，缓存一次都不会命中——这是最常见的翻车点。
  5. 我们的实测：30 篇、134 块，不开缓存输入 103017 token，开缓存后拆成写入 17340 加读取 60137，一次性成本降约 29%。**块切得越碎这个比例越高**，因为重复读的次数更多。
  6. 结论反直觉但很实用：一次性那笔是小钱，摊到 217 次查询就降到每次查询成本的一成以下；真正的长期账是每次查询多出来的那几十个 token（我们量到 +12.3%）。所以压成本的第一优先级不是压建索引，而是让块头别进上下文、别过长、别对全库无差别地生成。
  7. 最后一条是加分项：花这笔钱之前先确认你的评估环境**测得出**收益。我们的离线环境里向量路对召回的独立贡献实测为 0，所以它根本没法回答『块头对向量侧有没有用』——在这种环境里做的 A/B 会给你一个看起来有数字支撑的错误结论。

  How to reason about it · think before answering

  1. This checks whether you have actually done the arithmetic. Saying 'prompt caching makes it cheap' without knowing which line item it touches is a tell.
  2. Split the bill first: one-off = per-chunk input + output + full re-embedding; per-query = the header read twice, once by the reranker and once in the context. Keep them separate, because they scale with completely different things.
  3. The dominant term on the one-off side is how many times the same document is re-read. A doc split into n chunks is read n times. Prompt caching attacks exactly that: put the whole document first and mark it cacheable, pay a cache write once, then cache reads for the remaining n-1, typically an order of magnitude cheaper than input.
  4. Order matters. Caching is prefix-matched, so the document must come first and the chunk after. Put the varying part first and the prefix changes every call — zero cache hits. This is the most common way people get it wrong.
  5. Our measurement: 30 docs, 134 chunks. Without caching, 103017 input tokens; with caching, 17340 written plus 60137 read, cutting the one-off cost by roughly 29%. The finer the chunks, the bigger the saving, because re-reads multiply.
  6. The counter-intuitive part is the useful part: the one-off cost amortizes below 10% of per-query cost after about 217 queries. The lasting bill is the extra tokens every query carries (we measured +12.3%). So the first lever is not cheaper index building — it is keeping the header out of the context, keeping it short, and not generating it for the whole corpus indiscriminately.
  7. A bonus point: before spending any of it, confirm your evaluation setup can actually detect the benefit. In our offline harness the vector route contributed exactly zero unique answer documents, so it cannot answer whether headers help embeddings at all — an A/B run there hands you a wrong conclusion that looks numerically supported.

  答题要点

  • 把账拆成一次性（每块的输入输出 + 全量 embedding）和每次查询（块头在重排与上下文里各读一遍）两笔。
  • 一次性的大头是同一篇被重复读 n 遍；提示词缓存把它压成一次写入加 n-1 次读取。
  • 缓存按前缀匹配，整篇必须放在提示词最前面，块内容在后，顺序反了一次都不会命中。
  • 实测 30 篇 134 块，一次性成本降约 29%，块越碎省得越多。
  • 长期账在每次查询：块头别进上下文、控制长度、只对真正需要的文档生成。

  Key points

  • Split into one-off (per-chunk input/output plus re-embedding) and per-query (header read by both reranker and generator).
  • The one-off is dominated by re-reading each document n times; caching turns that into one write plus n-1 reads.
  • Caching is prefix-matched: the full document must come first, the chunk after, or you get zero hits.
  • Measured on 30 docs / 134 chunks, caching cut the one-off cost by about 29%, and finer chunks save more.
  • The lasting cost is per query: keep headers out of the context window, keep them short, and generate them selectively.