Interview Bank

Mastering Claude: From Conversation to Claude Code in 5 Days

D2 Long Documents, Multimodal Input, and a First Look at the API: Using Large Context, Saving Money With Prompt Caching, PDF and Image Input, Citation-Backed Answers; a Minimal Messages API Call

• Why is the context window called the scarcest resource in LLM applications? With million-token windows, does that still hold?为什么说上下文窗口是 LLM 应用里最稀缺的资源？窗口已经有一百万 token 了，这个说法还成立吗？

  Common in ChinaCommon overseasBasic#context-window#cost

  How to reason about it · think before answering

  1. The second sentence is the point. 'The window has a limit' is a dated answer; explain why scarcity survives large windows.
  2. Three causal chains: models are stateless so every request re-reads the whole input and bills it, a big window only solves fitting, not re-sending; longer context means more latency and diluted attention, so adherence to early instructions degrades as the window fills; and in agent workflows every file read and command output lands in the same window, filling it far faster than chat does.
  3. Conclusion: scarcity shifted from 'won't fit' to 'every token costs money and attention', so the discipline becomes active management — include only what is needed, cache the stable prefix, delegate research to subagents with their own context, and clear between tasks.
  4. Production math: a 60-page PDF is roughly 100k tokens; ten questions about it are a million input tokens; caching versus not caching is an order of magnitude apart.
  5. Follow-up: when should context accumulate? While deep in one complex problem where the history is still load-bearing; the test is whether the next step will use it.

  分析过程 · 先想清楚再作答

  1. 题眼在第二句。只答「窗口有上限」已经过时了，面试官想听的是「窗口变大之后为什么还稀缺」。
  2. 从三条因果链推：一、模型无状态，每次请求都把全部输入重读一遍，输入 token 按次计费——窗口大只解决了放得下，没解决每次都要重搬；二、上下文越长，延迟越高、注意力越稀释，模型对早期指令的遵守度会下降，也就是「性能随填充度下降」；三、Agent 场景里每读一个文件、每跑一条命令的输出都进同一个窗口，填得比聊天快得多。
  3. 结论：窗口大了，稀缺性从「放不下」变成了「每一 token 都在花钱和稀释注意力」，所以管理手段变成了主动管：只放必要的、把不变的缓存起来、把查资料的活派给独立上下文的子代理、该清就清。
  4. 生产视角：算一笔账——60 页 PDF 约 10 万 token，围着它问 10 个问题就是 100 万输入 token；不用缓存和不用缓存的差价是一个量级。
  5. 可预期的追问：那什么时候应该让上下文积累？在一个复杂问题里深挖时历史是有价值的；判据是「这段历史下一步还会不会用到」。

  Key points

  • Models are stateless: every request re-reads and bills the full input; a large window solves fitting, not re-sending
  • Longer context raises latency and dilutes attention; adherence to early instructions drops
  • Agent workflows dump every file read and command output into the same window
  • Tactics: include only what's needed, cache the stable prefix, isolate research in subagents, clear between tasks

  答题要点

  • 模型无状态，每次请求重读全部输入并计费；窗口大只解决放得下，不解决每次重搬
  • 上下文越长延迟越高、注意力越稀释，早期指令遵守度下降
  • Agent 场景每次读文件、跑命令的输出都进窗口，填得比聊天快得多
  • 对策：只放必要的、缓存不变前缀、用子代理隔离查资料、任务之间清空
• Where does prompt caching save money, when does it cost more, and how do you debug a zero cache-hit rate in production?prompt caching 省在哪？什么情况下反而不省？线上发现缓存命中率是零，你怎么排查？

  Common in ChinaCommon overseasIntermediate#prompt-caching#cost

  How to reason about it · think before answering

  1. Three questions, three layers: mechanism, boundaries, debugging. The third layer is what shows production experience.
  2. Mechanism: the cache matches the exact byte prefix from the start of the request to the cache_control marker (tools, then system, then messages). A hit bills that prefix at 0.1x input price; the write costs 1.25x (2x for the one-hour TTL).
  3. When it costs more: a prefix used only once (+25%); volatile content inside the prefix — timestamps, random ids, unsorted JSON, user names — so every call writes a cache nothing will read; a prefix below the minimum (1024 tokens on current flagship models, 4096 on Haiku 4.5) that silently never caches; requests spaced beyond the TTL.
  4. Debug order by likelihood: dynamic content at the head of system or tool definitions; model id mismatch between calls; prefix under the minimum; gap over five minutes; unstable tool ordering. The single signal is usage.cache_read_input_tokens greater than zero.
  5. Follow-up: where do breakpoints go? At the end of stable sections — tools, system, the long document, the second-to-last message in a multi-turn chat — at most four; a breakpoint on per-turn content is a wasted write.

  分析过程 · 先想清楚再作答

  1. 三问对应三层：原理、边界、排查。只答第一层是背文档，第三层才体现有没有真的上过线。
  2. 原理一句话：缓存匹配的是请求开头到 cache_control 标记为止的精确前缀（顺序是工具、system、messages），命中时这段只收正常输入价的 0.1 倍；代价是写入那一次收 1.25 倍（1 小时档 2 倍）。
  3. 不省的情况由此推出：同一前缀只用一次（多付 25%）；前缀里有每次都变的内容（时间戳、随机 id、未排序 JSON、用户名），导致每次都在写永远用不上的缓存；前缀短于最小门槛（主力模型 1024 token，Haiku 4.5 是 4096）根本不会缓存；两次请求间隔超过 TTL。
  4. 排查清单按发生概率排：一看 system 或工具定义开头有没有动态内容；二看两次请求的模型 id 是否一致；三看前缀长度是否过门槛；四看间隔是否超 5 分钟；五看工具列表顺序是否稳定。判据只有一个字段：usage.cache_read_input_tokens 是否大于 0。
  5. 可预期的追问：断点应该打在哪？不变的末尾——工具定义末尾、system 末尾、长文档末尾、多轮对话倒数第二条消息，最多四个；打在每轮都变的内容上等于白写。

  Key points

  • Matches the exact prefix (tools → system → messages up to the marker); hits bill 0.1x, writes 1.25x
  • Costs more when the prefix is used once, contains volatile content, is under the minimum length, or requests exceed the TTL
  • Debug: dynamic content, model mismatch, length, gap, tool ordering; verify via cache_read_input_tokens
  • Place breakpoints at the end of stable sections, at most four

  答题要点

  • 匹配精确前缀（工具 → system → messages 到标记为止）；命中 0.1 倍，写入 1.25 倍
  • 不省：前缀只用一次、前缀含动态内容、前缀短于最小门槛、间隔超过 TTL
  • 排查：动态内容、模型不一致、长度不够、间隔太久、工具顺序变了；看 cache_read_input_tokens
  • 断点打在不变部分的末尾，最多四个