Interview Bank

RAG in 14 Days: From Retrieval to Trustworthy Answers

D13 Going to Production: Incremental Sync and Deduplication, Permission-Based Filtering, Cache Layering, Tracing, and the Cost-Latency Ledger

• What can be cached in a RAG system, and what are the invalidation conditions for each?RAG 系统里有哪些东西可以缓存？各自的失效条件是什么？

  Common in ChinaCommon overseasIntermediate#caching#invalidation#cost-optimization

  How to reason about it · think before answering

  1. This looks like a giveaway and is actually a filter. 'Cache the question and answer' earns a third of the credit; the interviewer is waiting for the layering and the per-layer invalidation rules.
  2. Lead with a transferable rule: 'when must this be invalidated' is the same question as 'is that thing part of the key'. Leave something out of the key and changes to it will never invalidate the entry. With that rule the three layers derive themselves.
  3. Then go layer by layer. The answer layer maps a question to a final answer; its key needs the question, the permission scope, the index version, and the model plus prompt version. The retrieval layer maps a query to a hit list; its key needs the question, scope, topK, index version and embedding backend, but not the generation model. The embedding layer maps text to a vector; its key is just the text and the backend.
  4. Emphasize the counterintuitive part of the embedding layer: it is content-addressed, so the index version must not be in its key. Put it there and a single sync invalidates tens of thousands of vectors, which is exactly the full rebuild you added caching to avoid. This is the one layer that can live a long time, even on disk.
  5. Offer a concrete invalidation mechanism: version numbers rather than targeted deletion. Bump an index version whenever a sync actually changes something and old keys simply stop being computed. Targeted deletion would require enumerating which questions a change affected, and that list cannot be produced.
  6. Expected follow-up: can you give a real 'should have expired but didn't' case? Yes: an answer cache keyed only on the question. A document's limit changes from 200 MB to 500 MB, the index is updated, and the same question still returns 200 MB. Nothing errors; the log shows a clean cache hit. The same key also serves one department's answer to a user from another.

  分析过程 · 先想清楚再作答

  1. 这题看起来是送分题，实际是筛人题。答成「把问答结果缓存起来」只拿到三分之一，面试官等着听的是「分几层」和「各自什么时候失效」。
  2. 先给一条能迁移到别的题上的判断依据：**「什么时候必须失效」这个问题，等价于「key 里有没有把那样东西算进去」。** key 少放一样，那样东西变了缓存就不会失效。有了这条，三层的答案自己就长出来了。
  3. 然后逐层给：答案层缓存问题到最终答案，key 要有问题、权限范围、索引版本、模型与提示词版本；检索层缓存检索式到命中块列表，key 要有问题、权限范围、topK、索引版本、向量后端，但不需要模型；向量层缓存文本到向量，key 只有文本和向量后端。
  4. 重点讲向量层的反直觉之处：它是**内容寻址**的，文本没变、模型没变，向量就不会变，所以**不能把索引版本放进它的 key**。放进去的话一次同步就作废几万条向量，正好绕回全量重建——你加缓存想省的那笔钱又花回去了。这一层可以放很久甚至持久化。
  5. 给一个具体的失效手法：用**索引版本号**而不是精确删除。同步只要真的改动了索引就把版本号加一，旧 key 再也算不出来，自然没人读得到。精确删除要求你能列出「这次改动影响了哪些问题」，而那是列不出来的。
  6. 可预期的追问：能举一个「该失效却没失效」的真实例子吗？答：答案缓存的 key 只放了问题本身，文档里的上限从 200 MB 改成 500 MB、索引已经更新，再问同一个问题仍然返回 200 MB。它不报错，日志上是一次漂亮的缓存命中；同一个 key 还会让另一个部门的用户直接命中别人的答案。

  Key points

  • Three layers — answer, retrieval, embedding — with lifetimes orders of magnitude apart; treating them as one thing is the mistake.
  • The rule is that 'when must it expire' equals 'is it in the key'; anything left out of the key can never invalidate the entry.
  • The answer key carries question, permission scope, index version, model and prompt version; the retrieval key drops the model and adds topK and the embedding backend.
  • The embedding layer is content-addressed and keyed only on text plus backend; adding an index version turns every sync back into a full rebuild.
  • Version-based invalidation beats targeted deletion because you cannot enumerate which questions a given change affected.

  答题要点

  • 分三层：答案、检索、向量，三者的寿命差着数量级，不能当成一件事。
  • 判断依据是「什么时候必须失效」等价于「key 里有没有算进那样东西」，key 少一样就永远失效不了。
  • 答案层 key 要有问题、权限范围、索引版本、模型与提示词版本；检索层去掉模型、加上 topK 与向量后端。
  • 向量层是内容寻址的，key 只有文本与后端；把索引版本放进去会让每次同步都退化成全量重建。
  • 用索引版本号做失效比精确删除可靠，因为「这次改动影响了哪些问题」根本列不出来。