Interview Bank

Context Engineering in 5 Days

D3 Managing Context for Tool Results and Retrieval: Loading on Demand, Summarizing and Pruning, Structured Returns

• How do you choose between just-in-time loading and pre-inference retrieval, and how would you combine them?按需加载和预先检索怎么选？混合策略应该怎么搭？

  Common in ChinaCommon overseasIntermediate#retrieval#just-in-time#hybrid

  How to reason about it · think before answering

  1. This tests situational judgment. Calling just-in-time more advanced reads as trend-following, because pre-inference retrieval is genuinely better in many cases.
  2. Separate with three questions: can the needed material be scoped in advance (yes favors pre-retrieval), how fast does the material change (fast means indexes go stale, favoring just-in-time), and can it be fetched in one shot (multi-hop exploration forces just-in-time).
  3. Name the underlying difference: pre-retrieval hands the what-to-fetch decision to a retrieval algorithm and settles it before inference; just-in-time hands it to the model and spreads it across the run. The first is faster and more predictable, the second handles not knowing in advance.
  4. Conclusion is hybrid: preload the small, stable, always-relevant slice such as a project's standing instruction file, and use runtime search primitives for the rest. That gives a fast start without stale indexing, which is what coding agents converge on.
  5. Expect the follow-up on cost: just-in-time adds round trips and latency, and every fetched body stays in context consuming budget, so it must be paired with trimming.

  分析过程 · 先想清楚再作答

  1. 这题在考场景判断。答「按需加载更先进」的会被当成跟风，因为预先检索在很多场景里就是更好的选择，说不出它好在哪说明没做过。
  2. 怎么拆：用三个问题分开。需要的资料范围事先能不能确定（能就预先检索）、资料变化快不快（变得快索引一建就旧，偏按需加载）、一次能不能取完（要顺着线索翻好几层就只能按需）。
  3. 把两者的本质差别点出来：预先检索把「取什么」的决定权交给检索算法，在推理之前一次性做完；按需加载把这个决定权交给模型自己，在推理过程中分多次做。前者延迟低、可预测，后者能应付事先不知道要什么的情况。
  4. 结论是混合：把最稳定最常用的一小部分预先放进去（比如项目的常驻说明文件），其余靠运行时的搜索原语现取。这样既有起步速度，又不会被过期索引拖住。这也是编码类 Agent 的主流做法。
  5. 可预期的追问：按需加载的成本在哪？多了几轮往返，延迟更高，而且每一次取回的正文都会留在上下文里继续占预算——所以它必须和裁剪配套，取回来的东西该扔的时候要扔。

  Key points

  • Three questions: can scope be fixed in advance, how fast does the data change, and can it be fetched in one shot.
  • Pre-retrieval delegates the fetch decision to an algorithm before inference; just-in-time delegates it to the model during the run.
  • Most real systems are hybrid: preload the stable core, use runtime search for the rest, avoiding stale indexes.
  • Just-in-time costs round trips and latency, and fetched bodies keep consuming budget, so pair it with trimming.

  答题要点

  • 三个判断：范围能不能事先确定、资料变化快不快、一次能不能取完。
  • 预先检索把取什么的决定交给检索算法并在推理前做完；按需加载把它交给模型并分多次做。
  • 多数真实项目是混合：稳定常用的一小部分预加载，其余靠运行时搜索原语现取，避开索引过期。
  • 按需加载的代价是多轮往返与延迟，且取回的正文会继续占预算，必须和裁剪配套。