Interview Bank

From Frontend Engineer to Agent Engineer in 30 Days

D12 Long-Term Memory: pgvector, Embeddings, Chunking, the memory_search Tool

• Why does an agent need a separate long-term memory instead of stuffing all history into the context window?为什么 Agent 需要额外的长期记忆，而不是把历史全部塞进上下文？

  Common in ChinaCommon overseasBasic#long-term-memory#rag#cost

  How to reason about it · think before answering

  1. The tempting answer is 'the window is too small'. That is half right and it is the cheap half — windows keep growing, and the interviewer will ask what you would do at a million tokens.
  2. Separate the two problems first: context compression solves 'this turn does not fit in one session', long-term memory solves 'I cannot recall what was said last month'. One subtracts at request-assembly time, the other adds. Naming that distinction unprompted is where the signal is.
  3. Then quantify: 200 memories at roughly 400 tokens each is 80k tokens; at 0.15 USD per million input tokens that is 0.012 USD every single turn, about 0.24 USD per user per day at 20 turns. Retrieving the top 5 is 2k tokens, 0.0003 USD per turn — a 40x gap, and it repeats every turn.
  4. Give the reason that beats cost: irrelevant context lowers accuracy. If one of 200 memories is relevant, the other 199 are noise that pull the model toward answering something nobody asked. So even with an infinite free window, you would still retrieve rather than dump.
  5. Land on practice: distil cross-session user facts and preferences into standalone statements, store them as vectors, and inject the three to five most relevant per turn — the minimal form of RAG.
  6. Expect the follow-up: what belongs in long-term memory? Three tests — is it still needed across sessions, does it expire, can retrieval find it again. 'Lives in Shanghai' passes all three; 'shorten that paragraph' passes none.

  分析过程 · 先想清楚再作答

  1. 这题最容易答成「因为窗口装不下」。那只答对了一半，而且是不值钱的那一半——窗口一年比一年大，光靠这条理由，面试官会追问「等窗口到一百万 token 呢」，你就没词了。
  2. 先把两个问题拆开：上下文压缩解决的是「同一次会话里这一轮塞不下」，长期记忆解决的是「上个月说过的事想不起来」。前者在组装请求时做减法，后者做加法，触发时机、数据去向、失败后果都不同。能主动区分这两件事，是这题最大的区分度。
  3. 然后给成本账：200 条记忆、每条约 400 token 就是 8 万 token，按输入价 0.15 美元每百万 token 算，每一轮多付 0.012 美元；一天 20 轮就是 0.24 美元一个用户。只检索最相关的 5 条是 2000 token、每轮 0.0003 美元，差 40 倍。而且这笔钱是每轮重复付的，不是一次性的。
  4. 再给比钱更硬的理由：无关信息会降低命中率。200 条里跟这一轮相关的可能只有 1 条，剩下 199 条是噪声，模型会被带偏去回答一个用户没问的问题。**所以哪怕窗口无限大、token 免费，也该检索而不是全塞。** 这一句是这题的最优解。
  5. 落到做法上：把跨会话的用户事实与偏好抽成陈述句存进向量库，每轮按语义检索最相关的三五条注入请求——这就是 RAG 最小的一环。
  6. 可以预期的追问：什么信息该进长期记忆？答三问——跨会话之后还需要吗、会不会随时间失效、能不能靠检索捞回来。「用户住上海」三条都满足，「把刚才那段改成三句话」一条都不满足。

  Key points

  • Compression handles 'this turn does not fit'; long-term memory handles 'what did they say last month' — different problems, different machinery
  • Dumping everything costs on every turn: 200 memories is about 80k tokens and 0.012 USD per turn versus 0.0003 USD for five retrieved ones
  • The stronger reason is accuracy — irrelevant memories are noise, so you would retrieve even with an infinite window
  • Practice: distil cross-session facts into statements, embed them, inject the top three to five per turn
  • Admission test for a memory: still needed across sessions, does not expire, and is findable by retrieval

  答题要点

  • 压缩管「这一轮塞不下」，长期记忆管「上个月说过的事想不起来」，是两个问题、两套机制
  • 全塞的成本是每轮重复付的：200 条约 8 万 token，每轮多 0.012 美元；检索 5 条只要 0.0003 美元
  • 更硬的理由是准确率：无关记忆是噪声，会把模型带偏，所以窗口再大也该检索而不是全塞
  • 做法是把跨会话的事实抽成陈述句、向量化存储，每轮按语义检索最相关的三五条注入
  • 判断一条信息该不该进长期记忆：跨会话还需要吗、会不会失效、能不能被检索到