面试题库

30 天从前端工程师到 Agent 工程师

D15 多 Agent 模式全景（Router/Supervisor、Planner-Executor、Critic、Swarm、Blackboard）与何时不该用；LangGraph 入门

• 常见的多 Agent 协作模式有哪些？分别适合什么形状的任务？What are the common multi-agent collaboration patterns, and what shape of task suits each?

  国内高频海外高频基础#multi-agent#orchestration#architecture

  分析过程 · 先想清楚再作答

  1. 这题看似送分，其实在筛「背过名词」和「拆过系统」。只报五个名字最多拿及格分，面试官真正想听的是你用什么维度把它们区分开——有维度说明你能给没见过的架构归类，没维度说明你只是读过一篇综述。
  2. 给一个可复用的维度：模式的差别不在名字，在图的形状。盯四件事就够——有没有分叉（运行时三选一）、有没有扇出（同时交给多个人）、有没有汇合（多份产出合到一起）、有没有回边（可以打回重做）。
  3. 然后逐个落位：Router/Supervisor 只有分叉，一次只找一个专家，难点在判断该找谁；Planner-Executor 是扇出加汇合，适合一件事拆成几件、几件之间没有先后；Critic 是分叉加回边，适合对错有明确判据、且重做比发出去便宜的产出；Swarm 也是分叉加回边，但下一棒交给谁由当前这位自己决定；Blackboard 是扇出加汇合加回边，参与者互相不知道对方存在，只认公共状态。
  4. 主动指出 Critic 和 Swarm 的四个特征一模一样，区别落在「回边由谁决定」——Critic 是固定的评审节点在判，Swarm 是当前这位自己判。**主动承认自己的判据在哪里失效，比多背一个模式名更能加分**，因为它证明你真的用过这套维度而不是刚编出来。
  5. 每种模式还要配一句代价，这是区分度所在：Router 多一次路由调用的延迟；Planner-Executor 的并行会带来状态写冲突，字段必须配合并规则；Critic 的回路必须有次数上限，否则永远出不了稿；Swarm 事先不知道会走多少步，成本和延迟都难封顶；Blackboard 的终止条件最难写，容易谁都不接活或者反复触发。
  6. 可以预期的追问：生产上你最常用哪个？答 Router/Supervisor，理由是它的失败模式最好理解——路由判错了看一眼路由理由就知道，而且它是唯一一个能顺便省钱的模式，简单意图可以路由到便宜的小模型。

  How to reason about it · think before answering

  1. This looks like a giveaway but it separates people who memorised names from people who have split a system. Listing five names is a bare pass; the interviewer wants the axis you use to tell them apart, because an axis means you can classify an architecture you have never seen.
  2. Offer a reusable axis: the difference is not the name, it is the shape of the graph. Four questions suffice — is there a branch (pick one at runtime), a fan-out (hand it to several at once), a join (merge several outputs), a back edge (send it back for rework).
  3. Then place each one: Router/Supervisor is branch only, one specialist per turn, the hard part is deciding who; Planner-Executor is fan-out plus join, for work that splits into independent pieces; Critic is branch plus back edge, for output with a clear pass/fail test where redoing is cheaper than shipping; Swarm is also branch plus back edge, but the next hop is chosen by whoever holds the baton; Blackboard is fan-out plus join plus back edge, participants unaware of each other, reacting only to shared state.
  4. Point out yourself that Critic and Swarm score identically on all four, and that the real difference is who decides the back edge — a fixed reviewer node versus the current agent. Volunteering where your own criterion breaks down scores better than reciting one more pattern name, because it proves you have used the axis rather than invented it on the spot.
  5. Attach a cost to each: Router adds one routing call of latency; Planner-Executor's parallelism creates write conflicts so fields need merge rules; Critic loops need a hard retry cap or nothing ever ships; Swarm has no upfront bound on steps so cost and latency are hard to cap; Blackboard has the hardest termination condition and tends to either stall or re-trigger.
  6. Expect: which do you use most in production? Say Router/Supervisor, because its failure mode is the easiest to read — check the recorded routing reason — and because it is the one pattern that can save money, by routing simple intents to a cheaper model.

  答题要点

  • 先给维度再给名字：分叉、扇出、汇合、回边四个特征就能把五种模式分开
  • Router/Supervisor 只有分叉，一次只找一个专家，难点是判断该找谁
  • Planner-Executor 是扇出加汇合，适合拆成几件互不依赖的小任务再合成一份交付
  • Critic 是分叉加回边，适合对错有明确判据、重做比发出去便宜的产出，必须配打回次数上限
  • Swarm 与 Critic 的四个特征相同，区别在回边由谁决定；Blackboard 靠公共状态解耦，终止条件最难写
  • 每种模式配一句代价：多一次调用的延迟、并行的写冲突、回路的死循环、步数不封顶、终止条件难定

  Key points

  • Give the axis before the names: branch, fan-out, join and back edge separate all five patterns
  • Router/Supervisor is branch only — one specialist per turn, the hard part is choosing who
  • Planner-Executor is fan-out plus join — split into independent subtasks, then merge into one deliverable
  • Critic is branch plus back edge — for output with a clear pass/fail test, and it needs a hard retry cap
  • Swarm scores the same as Critic; the difference is who decides the back edge. Blackboard decouples via shared state and has the hardest termination condition
  • Pair each with a cost: extra call latency, parallel write conflicts, infinite review loops, unbounded step count, fuzzy termination
• LangGraph 里节点、边、状态分别扮演什么角色？如果不用框架，你自己会怎么实现？In LangGraph, what roles do nodes, edges and state play? If you had no framework, how would you implement it yourself?

  国内高频海外高频进阶#langgraph#orchestration#state-management

  分析过程 · 先想清楚再作答

  1. 题眼在后半句。只答三个概念的定义，面试官会认为你读过文档；能说出「不用框架会难受在哪」，才证明你知道框架替你解决了什么。这类题的通用解法是：先讲自己手写的版本，再讲框架把哪几处收敛了。
  2. 先给手写版：一个循环，里面一串条件判断决定下一步走哪，中间用一个大对象在各步之间传数据。写到第三个分支就会撞上三件事——两步都往同一个字段写，是覆盖还是追加，你要在每个分支里手写一遍合并逻辑；中间过程全在局部变量里，出错只能靠打印；进程一挂就从头重来，已经花掉的模型调用钱白付。
  3. 然后一一对上：节点是一个普通函数，读全量状态、返回只含改动字段的增量；边是节点之间的连接，无条件边写死顺序，条件边在运行时决定去哪；状态是一张字段表，每个字段是一条独立通道，通道上挂着合并规则。
  4. 重点讲第三条，因为它是最容易被略过、也最值钱的一条：**合并规则是声明在字段上的，不是写在节点里的**。这意味着新增节点时不需要考虑「我该怎么和别人的写入合并」，字段自己知道；也意味着并行写同一个字段时行为是确定的，而不是取决于谁先返回。
  5. 配两个具体的坑，证明你真跑过：一是在节点里原地修改状态（比如直接往数组里 push）会绕过合并规则，单线程时察觉不到，并行时变成偶发覆盖；二是加了节点没连边不会报错，表现只是那个节点永远不执行，只能靠逐节点追踪发现。
  6. 可以预期的追问：那你为什么不直接自己写？答：三要素本身很轻，核心逻辑几十行就能手写出来——框架真正值钱的是检查点与恢复、并行执行、以及每一步的可观测，这三样自己写的成本远高于三要素本身。顺带说明 Java 和 Swift 没有官方 LangGraph，真要在这两门语言里做，就是把这三要素手写一遍。

  How to reason about it · think before answering

  1. The hinge is the second half. Defining the three concepts only proves you read the docs; explaining what hurts without a framework proves you know what it buys you. The general move for this family of questions is: describe your hand-rolled version first, then name what the framework collapsed.
  2. Hand-rolled version: a loop, a chain of conditionals picking the next step, and one big object carrying data between steps. By the third branch you hit three walls — when two steps write the same field, is it overwrite or append, and you hand-write that merge in every branch; intermediate state lives in local variables so debugging means print statements; a crash restarts from zero and the model calls you already paid for are wasted.
  3. Then map them: a node is an ordinary function that reads the whole state and returns a delta containing only what it changed; edges connect nodes, unconditional ones fix the order and conditional ones decide at runtime; state is a table of fields where each field is its own channel carrying a merge rule.
  4. Dwell on the third, which is the most skipped and most valuable point: the merge rule is declared on the field, not written inside the node. Adding a node therefore requires no thought about how to combine with other writers, and parallel writes to one field behave deterministically instead of depending on who returns first.
  5. Add two concrete traps to show you have actually run this: mutating state in place inside a node bypasses the merge rule — invisible single-threaded, an intermittent overwrite once things run in parallel; and adding a node without wiring an edge raises no error at all, it simply never executes, which only per-node tracing reveals.
  6. Expect: so why not just write it yourself? Because the three primitives are genuinely light — a few dozen lines. What the framework actually sells is checkpointing and recovery, parallel execution, and per-step observability, all of which cost far more to build than the primitives. Mention too that there is no official LangGraph for Java or Swift, so in those languages you do hand-roll exactly these three.

  答题要点

  • 节点是普通函数：读全量状态，返回只含改动字段的增量，不在节点里原地改状态
  • 边决定执行顺序：无条件边写死，条件边在运行时决定下一步去哪（Supervisor 就靠它）
  • 状态是一张字段表，每个字段一条通道，通道上挂合并规则——规则声明在字段上而不是写在节点里
  • 不用框架会撞三堵墙：合并逻辑在每个分支手写一遍、中间过程只能靠打印、进程挂了从头重来
  • 两个真实的坑：原地改状态绕过合并规则（并行时偶发覆盖）、加了节点没连边不报错只是永不执行
  • 框架真正值钱的不是这三要素，而是检查点恢复、并行执行和逐步可观测

  Key points

  • A node is a plain function: read the full state, return a delta of changed fields only, never mutate in place
  • Edges set execution order: unconditional edges are fixed, conditional edges decide the next hop at runtime — that is what a supervisor uses
  • State is a table of fields, each field a channel carrying a merge rule declared on the field rather than inside nodes
  • Without a framework you hit three walls: hand-written merges in every branch, no visibility into intermediate steps, and full restart after a crash
  • Two real traps: in-place mutation bypasses the merge rule and causes intermittent overwrites under parallelism; an unwired node raises no error, it just never runs
  • What the framework really sells is checkpoint recovery, parallel execution and per-step observability — not the three primitives themselves
• 从单 Agent 升级到多 Agent，通常是被什么信号触发的？升级之后系统会多付出什么？What signals typically trigger the move from a single agent to a multi-agent system, and what does the upgrade cost you?

  国内高频海外高频进阶#multi-agent#cost#architecture

  分析过程 · 先想清楚再作答

  1. 这题考的是「你是被业务逼着拆的，还是照着博客拆的」。答「业务变复杂了」等于没答，面试官要的是**可观测的信号**：什么现象出现时你才动手。
  2. 给五个按出现顺序排的信号：一是提示词开始互相打架（加一条规则，另一个指标就掉）；二是工具列表长到自己都要查文档；三是某一步的失败需要单独处理，不该整轮重来；四是想给某一步单独换模型；五是评估颗粒度不够，只能整体打分好或不好。
  3. 第四个信号要展开讲，它是唯一一个反常识的：多 Agent 通常更贵，但按步换模型是它唯一能省钱的场景——分诊这种短判断走便宜的小模型，拟方案走大模型。单 Agent 做不到按步换模型。这一条在面试里是明显的亮点。
  4. 然后主动给代价，不给代价的回答会被当成布道：延迟按步数乘倍数（原来两秒变六秒，而用户耐心大约三秒）；成本按调用次数线性涨，因为每一步都要把当前状态重新塞进上下文，典型是三倍；调试难度按状态维度涨，出错要同时回答路由对不对、每个子 Agent 拿到的状态对不对、合并有没有互相覆盖。
  5. 再补一句反向判断，证明你不是无脑拆：工具太多的第一反应应该是合并工具、收敛描述，拆 Agent 是第二反应；质量差的第一反应应该是把单 Agent 版本调到最好，那个版本还会成为多 Agent 的对照基线。
  6. 可以预期的追问：拆完怎么证明比原来好？答：留住单 Agent 版本当基线，用同一批标准样本集跑 A/B，比准确率也比每次对话的成本与延迟。说不出对照基线的人，通常也说不清自己为什么拆。

  How to reason about it · think before answering

  1. This question tests whether business pain forced the split or a blog post did. Answering the business got complex is a non-answer; the interviewer wants observable signals — what symptom made you act.
  2. Give five, in the order they usually appear: prompts start fighting each other (add one rule, another metric drops); the tool list grows until you need the docs yourself; one step's failure needs isolated handling instead of redoing the whole turn; you want a different model for one specific step; and evaluation granularity is too coarse to say more than good or bad.
  3. Expand on the fourth, the counter-intuitive one: multi-agent is usually more expensive, but per-step model selection is the one case where it saves money — a short triage decision on a cheap small model, a drafting step on a larger one. A single agent cannot swap models per step. This lands well in interviews.
  4. Then volunteer the costs, or the answer reads as evangelism: latency multiplies by step count (two seconds becomes six, while user patience is about three); cost grows linearly with calls because every step re-sends the current state as context, typically three times; and debugging cost grows with state dimensions, since a failure now requires checking routing, each sub-agent's input state, and whether merges overwrote each other.
  5. Add the reverse check to show you are not splitting reflexively: too many tools should first prompt consolidation and tighter descriptions, with splitting as the second response; poor quality should first prompt tuning the single-agent version to its best, which then becomes the baseline the multi-agent version is measured against.
  6. Expect: how do you prove the split helped? Keep the single-agent version as a baseline and A/B both against the same golden set, comparing accuracy alongside per-conversation cost and latency. People who cannot name a baseline usually cannot explain why they split either.

  答题要点

  • 五个可观测信号：提示词互相打架、工具多到要查文档、某一步需要独立重试、想按步换模型、评估颗粒度不够
  • 按步换模型是多 Agent 唯一能省钱的场景：短判断走小模型、拟方案走大模型，单 Agent 做不到
  • 代价一：延迟按步数乘倍数，两秒变六秒，而用户对客服机器人的耐心大约三秒
  • 代价二：成本线性涨，每一步都要把状态重新塞进上下文，典型是原来的三倍
  • 代价三：调试难度按状态维度涨，所以多 Agent 和链路追踪必须一起上
  • 反向判断：工具多先合并再拆分，质量差先把单 Agent 调到最好——那个版本还是多 Agent 的对照基线

  Key points

  • Five observable signals: prompts fighting each other, a tool list you must look up, one step needing isolated retries, wanting a different model per step, and evaluation too coarse to act on
  • Per-step model selection is the only case where multi-agent saves money: small model for triage, larger model for drafting — impossible in a single agent
  • Cost one: latency multiplies with step count, two seconds becomes six, while patience for a support bot is about three
  • Cost two: spend grows linearly with calls since every step re-sends state as context, typically three times the original
  • Cost three: debugging cost grows with state dimensions, so multi-agent and tracing have to ship together
  • Reverse check: consolidate tools before splitting, and tune the single agent to its best first — that version becomes your baseline
• 什么情况下不应该引入多 Agent 系统？请给出可操作的判据，而不是「视情况而定」。When should you not introduce a multi-agent system? Give operational criteria, not it depends.

  国内高频海外高频深入#multi-agent#architecture#trade-offs

  分析过程 · 先想清楚再作答

  1. 这是本组最有区分度的题，因为它反着问。绝大多数候选人会顺着「多 Agent 很强大」讲下去，而面试官问这题正是想找那个会说不的人——**在真实团队里，拦住一次不必要的架构升级，价值高于实现三个模式**。
  2. 先给结论式的默认值：默认答案是不拆。然后给三条判据，命中任意一条才拆——一是单个 Agent 的系统提示词里出现了互斥的行为要求（既要严格核对退款规则又要热情挽留，这两条不是难写，是不可能同时最优）；二是工具数量超过模型能稳定选对的规模（经验线大约八个，超线的第一反应是合并工具而不是拆 Agent）；三是某一步需要独立的失败与重试语义。三条都不命中，单 Agent 加几个工具就够。
  3. 接着点名最常见的错拆：把提示词问题当成架构问题。「回答质量不好，所以拆成三个 Agent」——质量差有九成来自提示词含糊、工具描述互相干扰、上下文塞了无关历史，这三样拆完一样存在，只是分散到三个地方更难查。**拆 Agent 解决的是职责冲突，不是能力不足。**
  4. 再补两类明确不该拆的场景：一是低延迟要求的场景，多一跳就多一次模型往返，对语音或实时补全这类交互直接不可用；二是只读的简单查询链路，三五个工具的客服助手拆了只是把一次调用变成三次，准确率不会涨、账单会涨。
  5. 然后给一条可执行的验证路径，这是加分项：任何拆分都先留住单 Agent 版本当对照基线，用同一批标准样本集跑 A/B，同时比准确率、每次对话成本和延迟。**拿不出对照基线的架构升级，等于没有证据的重构。**
  6. 可以预期的追问：那如果老板就是要求上多 Agent 呢？答：那就把它当成一个可回退的实验来做——先按判据拆最有把握的那一刀（通常是互斥规则那一条），保留基线，两周后拿数据说话。这个回答同时展示了技术判断和沟通方式，比硬顶或硬上都好。

  How to reason about it · think before answering

  1. This is the highest-signal question in the set because it is asked in reverse. Most candidates keep selling how powerful multi-agent is, while the interviewer is looking for someone who will say no — on a real team, blocking one unnecessary architecture upgrade is worth more than implementing three patterns.
  2. Lead with the default: do not split. Then give three criteria, any one of which justifies splitting — the system prompt contains mutually exclusive behavioural requirements (strictly enforce refund rules while also warmly retaining the customer; these are not hard to write, they are impossible to optimise together); the tool count exceeds what the model picks reliably (roughly eight as a rule of thumb, and the first response to crossing it is consolidating tools, not splitting agents); or one step needs its own failure and retry semantics. None of the three, and a single agent with a few tools is enough.
  3. Then name the most common bad split: treating a prompt problem as an architecture problem. Quality is poor, so we split into three agents — but nine times out of ten poor quality comes from vague prompts, tool descriptions that interfere with each other, or irrelevant history in the context. All three survive the split and are now harder to find. Splitting fixes conflicting responsibilities, not weak capability.
  4. Add two scenarios that clearly should not split: latency-sensitive interactions, where each extra hop is another model round trip and voice or realtime completion becomes unusable; and read-only lookup flows, where a support assistant with three or four tools gains no accuracy from splitting and simply triples the bill.
  5. Then offer an executable verification path, which earns points: keep the single-agent version as a baseline for any split and A/B both against the same golden set, comparing accuracy, per-conversation cost and latency together. An architecture upgrade with no baseline is a refactor with no evidence.
  6. Expect: what if your manager insists on multi-agent? Frame it as a reversible experiment — make the one cut you are most confident in (usually the conflicting-rules criterion), keep the baseline, and bring data in two weeks. That answer shows technical judgement and a way to disagree without stonewalling.

  答题要点

  • 默认答案是不拆；三条判据命中任意一条才拆：提示词有互斥要求、工具超过约八个的告警线、某一步需要独立的失败与重试语义
  • 工具太多的第一反应是合并工具与收敛描述，拆 Agent 是第二反应
  • 最常见的错拆是把提示词问题当架构问题——质量差多半来自提示词含糊、工具描述干扰、上下文塞了无关历史，拆完这三样照旧存在
  • 明确不该拆：低延迟交互（每多一跳就多一次模型往返）、只读的简单查询链路（准确率不涨、账单涨）
  • 任何拆分都要留单 Agent 版本当对照基线，用同一批标准样本集比准确率、成本和延迟
  • 代价要说出口：延迟按步数乘倍数、成本约三倍、调试要同时排查路由与状态合并

  Key points

  • Default to not splitting; split only if one of three criteria holds: mutually exclusive prompt requirements, tool count past the roughly-eight warning line, or a step needing its own failure and retry semantics
  • Too many tools should first trigger tool consolidation and tighter descriptions; splitting agents is the second response
  • The most common bad split is treating a prompt problem as an architecture problem — vague prompts, interfering tool descriptions and irrelevant history all survive the split
  • Clear do-not-split cases: latency-sensitive interactions where every hop adds a model round trip, and read-only lookup flows where accuracy does not move but the bill does
  • Always keep the single-agent version as a baseline and compare accuracy, cost and latency on the same golden set
  • Say the costs out loud: latency multiplies with steps, spend roughly triples, and debugging now spans routing plus state merging

D16 Supervisor 动态路由：structured output 路由、override、routingReason

• Supervisor 模式里的路由决策一般怎么实现？请说说这个节点该做什么、不该做什么。How is the routing decision usually implemented in a supervisor pattern? What should that node do, and what should it not do?

  国内高频海外高频基础#multi-agent#routing#langgraph

  分析过程 · 先想清楚再作答

  1. 这是一道送分题，但送分题最容易答成「让一个 Agent 决定下一步找谁」这种复述题面的话。区分度在后半句：你能不能说清这个节点的职责边界。
  2. 先给机械原理：Supervisor 是图里的一个普通节点，它读状态、调一次模型、只写两个字段——交给谁（route）和为什么这么判（routingReason）；真正的分叉发生在它后面那条条件边上，边上挂一个选择函数，把 route 翻译成下一个节点名。
  3. 再划边界，这是拿分的地方：Supervisor 不回答用户的问题、不调业务工具、不产生副作用。它只做选择题，所以可以配一个更便宜的小模型，输入通常只有系统提示词加最后一两句话。
  4. 还有一条边界更容易被忽略：**选择函数里不要再调模型**。判断已经在 Supervisor 节点里做完并落进状态了，选择函数只做翻译。把模型调用塞进选择函数，同一份状态每次可能跳到不同的节点，图就不可复现，后面做检查点重放和评估都会失真。
  5. 最后补一句「一次只派一个人」：Supervisor 解决的是「交给谁」，不解决「一件事要拆成几件、还得有人验收」。后者是 Planner-Executor-Critic 的活。能主动划出这条线，面试官会认为你见过真实系统的边界。
  6. 可以预期的追问：那三个子 Agent 的名单从哪来、加一个新的要改几处？答案是名单应该是单一真相来源——枚举定义、schema、条件边的映射表都从它生成，加一个子 Agent 只改一处，其余地方编译期报错提醒你。

  How to reason about it · think before answering

  1. This is a warm-up question, and warm-ups are where people lose points by restating the prompt: an agent decides who goes next. The discriminator is the second half — can you state the node's responsibility boundary?
  2. Start with the mechanics: the supervisor is an ordinary node. It reads state, makes one model call, and writes exactly two fields — the route and the reason for it. The actual branching happens on the conditional edge after it, whose selector function maps the route to the next node name.
  3. Then draw the boundary, which is where the points are: the supervisor never answers the user, never calls business tools, and produces no side effects. It only takes a multiple-choice test, so it can run on a cheaper small model with a short input.
  4. One boundary people miss: do not call the model inside the selector function. The judgement was already made and stored in state; the selector only translates. Calling a model there makes the same state jump to different nodes across runs, which destroys reproducibility and breaks checkpoint replay and evaluation later.
  5. Close with one-at-a-time: a supervisor answers who takes this, not how to split a task and who reviews the output. That second problem belongs to planner-executor-critic. Drawing that line yourself signals you have seen a real system.
  6. Expect: where does the list of sub-agents live, and how many places change when you add one? Answer that the list should be a single source of truth — the enum, the schema, and the edge mapping all derive from it, so adding an agent is one edit and everything else fails at compile time.

  答题要点

  • Supervisor 是图里的一个普通节点：读状态、调一次模型、只写 route 与 routingReason 两个字段
  • 真正的分叉在它后面的条件边上：选择函数把 route 翻译成下一个节点名，映射表要写全
  • 职责边界：不回答用户、不调业务工具、不产生副作用，因此可以单独配一个更便宜的小模型
  • 选择函数里不能调模型，否则同一份状态每次跳的节点不同，图不可复现，检查点重放与评估都会失真
  • Supervisor 一次只派一个人，只解决「交给谁」；拆任务与验收是 Planner-Executor-Critic 的职责

  Key points

  • The supervisor is an ordinary node: read state, one model call, write only the route and the routing reason
  • Branching lives on the conditional edge after it — a selector maps the route to a node name, and the mapping table must be exhaustive
  • Boundary: it never answers the user, calls no business tools, has no side effects, so it can run on a cheaper small model
  • Never call a model inside the selector, or the same state jumps to different nodes across runs and replay and evaluation both break
  • A supervisor dispatches one agent at a time and only answers who takes this; splitting and reviewing belong to planner-executor-critic
• 为什么要让模型输出 structured output 而不是自然语言来做路由？自然语言到底差在哪？Why use structured output rather than natural language for routing? What exactly goes wrong with free text?

  国内高频海外高频进阶#structured-output#routing#reliability

  分析过程 · 先想清楚再作答

  1. 这题最容易答成「结构化更规范、更好解析」——这是形容词，不是理由。面试官想听的是一个具体的失败场景，最好是你真的调过的那种。
  2. 把最锋利的一刀先亮出来：**自然语言路由的失败是静默的**。模型回「我觉得这个可以让查订单的同事看一下」，它其实判对了，但说的是人话不是 id，正则匹配不上就落进兜底，日志里只留下一个 smalltalk。模型是对的、解析是错的，而它和「模型判错了」在日志里长得一模一样。你会去调提示词，调两天才发现问题在那三行正则。
  3. 然后给三个漏洞，一条一条对上结构化输出解决了什么：输出会漂移（今天回「订单查询」明天回「查订单」，正则永远追不上，模型小版本升级你就掉准确率）；没有置信度（自然语言里没有「我有多大把握」这个信息，你没法区分它很确定还是在猜）；拼错或自造的路由名要到运行时才炸（枚举是一道编译期就存在的闸门）。
  4. 接着说清机制，别停在「用 zod 更规范」：把 schema 发进请求（response_format 里的 json_schema），模型的解码过程被枚举约束；回来之后**用同一份声明再校验一遍**。一份声明两用，请求与校验不会漂移。
  5. 关键的反直觉点，答到这里就拉开差距了：**结构化输出不等于不用校验**。不是所有网关、所有模型都严格执行 schema，降级到备用模型时更说不准。所以解析函数的返回类型应该是「一段待校验的东西」，而不是「已经是 RouteDecision」。
  6. 可以预期的追问：那不支持 json_schema 的模型怎么办？答案是退回「few-shot 加严格提示词加自己校验」，闸门仍然在你的枚举校验那一步——真正兜底的从来不是模型的自觉，是你的解析层。

  How to reason about it · think before answering

  1. The trap is answering structured output is cleaner and easier to parse. Those are adjectives, not reasons. The interviewer wants a concrete failure you have actually debugged.
  2. Lead with the sharpest point: free-text routing fails silently. The model replies I think the order desk should look at this — it judged correctly, but it spoke prose, not an id. Your regex misses, you fall through to the default, and the log shows only smalltalk. A correct model with a broken parser looks exactly like a wrong model, so you spend two days tuning a prompt that was never the problem.
  3. Then list three holes and map each to what structured output fixes: wording drifts across versions so regexes never catch up; there is no confidence signal, so you cannot tell certainty from guessing; and an invented route name only explodes at runtime, whereas an enum is a gate that exists before the request is even sent.
  4. Explain the mechanism rather than stopping at zod is nicer: send the schema in the request (response_format with a json_schema), so decoding is constrained by the enum, then validate the response with the same declaration. One declaration used twice means request and validation cannot drift apart.
  5. The counterintuitive point that separates candidates: structured output does not remove the need to validate. Not every gateway or model enforces the schema strictly, and a fallback model may not at all. Your parse function should return something-to-be-validated, not an already-typed decision.
  6. Expect: what if the model does not support json_schema? Fall back to few-shot plus a strict prompt plus your own validation. The real gate was never the model's discipline; it is your parsing layer.

  答题要点

  • 自然语言路由的失败是静默的：模型判对了但说的是人话，正则匹配不上就落兜底，和判错在日志里完全一样
  • 三个漏洞：措辞会漂移（正则追不上）、没有置信度（分不清确定与猜）、自造的路由名要到运行时才炸
  • 机制是一份声明两用：schema 随请求发出去约束解码，回来后用同一份声明校验，请求与校验不会漂移
  • 枚举是编译期就存在的闸门，把「拼错的路由名」从线上事故降级成一次解析失败
  • 结构化输出不等于不用校验：网关和降级模型未必严格执行 schema，解析函数的返回类型应该是「待校验」而不是「已经是」

  Key points

  • Free-text routing fails silently: a correct judgement in prose misses your regex and falls through, looking identical to a wrong judgement in the logs
  • Three holes: wording drifts, there is no confidence signal, and invented route names only fail at runtime
  • One declaration used twice: the schema constrains decoding in the request and validates the response, so the two cannot drift
  • An enum is a gate that exists before the call, turning a misspelled route from an incident into a parse failure
  • Structured output does not remove validation — gateways and fallback models may not enforce the schema, so parsing must return an unvalidated value
• 路由不确定或者路由错误时，系统应该怎么兜底？阈值该怎么定？How should the system handle an uncertain or wrong routing decision, and how do you pick the threshold?

  国内高频海外高频深入#routing#fallback#reliability

  分析过程 · 先想清楚再作答

  1. 题眼在「不确定」和「错误」是两件事。多数人只答重试或人工接管，那是把两个问题揉成一个。区分度在于你能不能先给出一条价值判断，再给策略。
  2. 先立论：**路由到错的子 Agent，比路由失败糟糕得多**。失败你至少知道自己失败了，可以追问一句；错了，接手的子 Agent 完全不知道自己接错了活，会用笃定的语气给出一个格式完整的错误答案，用户不会怀疑，会照着去操作。一个自信的错误答案比一句「我没听清」贵一百倍。
  3. 再给可执行的策略，数字要具体：模型给的置信度低于 0.6，或者路由名不在合法名单里，一律落到兜底的 smalltalk，并在 routingReason 里打上 fallback 前缀加原因码（低置信度、未知路由、结构非法各一种）。兜底那位的人设是「信息不足先追问一句缺的关键信息，不要猜」——兜底的本质是把不确定性还给用户。
  4. 阈值怎么定这一问是重点，别背数字：**取决于两类错误哪一类更贵**。客服场景里多问一句只是用户小小的不耐烦，派错可能变成一条错误的退款承诺，所以宁可保守取 0.6；内部工具型 Agent 里多问一句反而更烦人，阈值就该放低。再补一句可落地的定法：拿标准样本集扫一遍，画出不同阈值下的误派率与追问率，选拐点。
  5. 必须点破的一个坑：**置信度是模型自己报的，它不是概率**。模型说 0.9 不代表有九成对。它只是同一模型、同一提示词下相对可用的排序信号，只能当闸门用，不能拿去算期望值。真正的准确率要靠离线评估去量。
  6. 可以预期的追问：兜底会不会把问题掩盖掉？答案是不会，前提是你记了原因码——把一周内落进兜底的请求按原因分组，能直接看出分诊提示词缺了哪一类描述。兜底是止血，原因码才是治本的输入。

  How to reason about it · think before answering

  1. The hinge is that uncertain and wrong are two different failures. Most candidates answer retry or escalate to a human, collapsing both into one. The discriminator is stating a value judgement before giving a policy.
  2. The claim first: routing to the wrong sub-agent is far worse than failing to route. A failure announces itself and lets you ask a clarifying question. A wrong route does not — the receiving agent has no idea it got the wrong job and will produce a confident, well-formatted, wrong answer that the user will act on. A confident wrong answer costs a hundred times more than I did not catch that.
  3. Then give a concrete policy with real numbers: if the model's confidence is below 0.6, or the route name is not in the allowed list, fall back to the small-talk agent and stamp the reason with a fallback prefix plus a cause code (low confidence, unknown route, invalid shape). The fallback agent's job is to ask for the one missing detail rather than guess — falling back means handing the uncertainty back to the user.
  4. The threshold question is the real test, so do not recite a number: it depends on which error is more expensive. In customer support one extra question costs mild annoyance while a misroute can become a wrong refund promise, so stay conservative. For an internal tool the extra question is the bigger cost, so lower it. Then give a method: sweep thresholds over a golden set, plot misroute rate against clarification rate, and pick the knee.
  5. Name the trap: the confidence number is self-reported and is not a probability. Nine tenths does not mean nine in ten are right. It is a usable ranking signal within one model and one prompt — good as a gate, useless for expected-value math. Real accuracy comes from offline evaluation.
  6. Expect: does falling back just hide the problem? Not if you record cause codes. Group a week of fallbacks by cause and you can see exactly which intent the routing prompt fails to describe. The fallback stops the bleeding; the cause code is what fixes it.

  答题要点

  • 先分清两件事：路由失败可以追问，路由错误会让子 Agent 自信地给出错误答案，后者贵得多
  • 策略：置信度低于 0.6 或路由名不在名单里，一律落兜底的 smalltalk，并在 routingReason 打上 fallback 前缀加原因码
  • 兜底不是随便找个人接，而是把不确定性还给用户——兜底那位应当追问缺失的关键信息而不是猜
  • 阈值取决于两类错误哪一类更贵：客服场景多问一句便宜、派错很贵，所以保守；定法是拿标准样本集扫阈值找拐点
  • 置信度是模型自报的，不是概率，只能当闸门用；真正的准确率要靠离线评估量
  • 落兜底时记原因码，按原因分组就能看出分诊提示词缺了哪一类描述

  Key points

  • Separate the two: a failed route can ask a clarifying question, a wrong route produces a confident wrong answer, and the second is far costlier
  • Policy: confidence below 0.6 or a route outside the allowed list falls back to small talk, stamped with a fallback prefix and a cause code
  • Falling back is not picking someone at random — the fallback agent asks for the missing detail instead of guessing
  • The threshold depends on which error costs more; sweep it over a golden set and pick the knee between misroutes and clarifications
  • Self-reported confidence is not a probability — use it as a gate only, and measure real accuracy offline
  • Record cause codes on every fallback; grouping them shows which intent the routing prompt fails to describe
• routingReason 这类调试信息在生产系统里有什么价值？只是打日志而已吗？What is a field like routingReason actually worth in production? Is it just logging?

  国内高频海外高频进阶#observability#routing#debugging

  分析过程 · 先想清楚再作答

  1. 这题看着像水题，其实在筛「有没有真的排查过线上问题」。答「方便调试」就结束的人，基本没值过班。
  2. 先给一条不可回避的事实：**路由决策是模型做的，而模型不可复现**。同一句话下次未必给同样的判断，你没法重跑一遍去看「当时是怎么想的」。所以理由必须在当时就写下来，否则那次判断永远丢了。这一条把 routingReason 从「日志」抬到了「唯一的审计证据」。
  3. 然后给三个具体用途，每个都要能落地：一是把「模型判错了」和「解析或兜底出错了」分开，前缀写成 fallback 加原因码，一眼就能分辨；二是攒下一版提示词的素材，把一周内落进兜底的请求按原因分组，会看到集中的几类意图缺描述；三是它是离线评估的输入——标准样本集要评的不只是最终回答，还有分诊准不准，而这件事只有当时记了判断和理由才评得了。
  4. 写法上有个细节值得主动说：**结构化的壳加自然语言的芯**。前缀（fallback 加原因、override 加目标）用来聚合统计，后面那句人话用来看具体这一单。整条都写成自然语言，就退回成本章批判的那种东西了。
  5. 再补一条容易被忽略的：兜底和人工改派都不要擦掉模型的原判，原样拼进理由里。否则一周后没人说得清这一单是模型判错了还是本来就被人改过——多写二十个字符，省掉一次翻遍代码的排查。
  6. 可以预期的追问：这些字段会不会带来隐私或成本问题？答案是会，所以理由里只写判断依据不写用户原文，长度设上限（比如 120 字），并且和链路追踪共用同一个 run 标识，别另起一套。

  How to reason about it · think before answering

  1. This looks like a throwaway question but it screens for whether you have ever been on call. Anyone who stops at it helps with debugging has not.
  2. Start with the fact you cannot design around: the routing decision is made by a model, and models are not reproducible. The same sentence may be judged differently next time, so you cannot re-run to see what it was thinking. The reason must be captured at decision time or it is gone forever — that is what turns this field from a log line into the only audit evidence you have.
  3. Then give three concrete uses. One, it separates a wrong model judgement from a parsing or fallback problem, provided the prefix carries a cause code. Two, it is raw material for the next prompt revision: group a week of fallbacks by cause and the missing intent descriptions jump out. Three, it feeds offline evaluation — a golden set should score routing accuracy, not just the final answer, and that is only scorable if the decision and its reason were recorded.
  4. Mention the shape: a structured prefix wrapping a human sentence. The prefix (fallback plus cause, override plus target) is what you aggregate on; the sentence is what you read for one specific case. Making the whole field prose puts you right back in the failure mode this chapter argues against.
  5. Add the detail people skip: neither a fallback nor a human override should erase the model's original judgement — carry it into the reason. Otherwise nobody can later tell whether the model got it wrong or a human redirected it. Twenty extra characters save an afternoon of archaeology.
  6. Expect: do these fields create privacy or cost problems? Yes, so record the basis for the decision rather than the user's raw text, cap the length, and reuse the same run identifier as your tracing instead of inventing a parallel one.

  答题要点

  • 路由决策由模型做出且不可复现，理由必须在当时写下来，否则那次判断永远丢了——它是唯一的审计证据
  • 用途一：把「模型判错」和「解析或兜底出错」分开，靠 fallback 加原因码一眼分辨
  • 用途二：把一周内落进兜底的请求按原因分组，直接得到下一版分诊提示词该补什么
  • 用途三：它是离线评估的输入，分诊准确率这个指标只有记了当时的判断与理由才评得了
  • 写法是结构化的壳加自然语言的芯：前缀用于聚合统计，人话用于看具体这一单
  • 兜底与人工改派都要保留模型原判；理由只写判断依据不写用户原文，长度设上限，并复用链路追踪的 run 标识

  Key points

  • The decision comes from a model and is not reproducible, so the reason must be captured at decision time — it is the only audit evidence you get
  • Use one: it separates a wrong model judgement from a parsing or fallback failure, via a cause code in the prefix
  • Use two: grouping a week of fallbacks by cause tells you exactly what the next routing prompt is missing
  • Use three: it feeds offline evaluation, since routing accuracy can only be scored if the decision and reason were recorded
  • Shape it as a structured prefix around a human sentence: aggregate on the prefix, read the sentence for one case
  • Keep the model's original judgement through fallbacks and overrides; store the basis rather than raw user text, cap the length, and reuse the tracing run id

D17 Planner–Executor–Critic + 共享工作区：workspace state、toolBudget、并行 fan-out、review 回路

• Planner-Executor-Critic 这种结构解决了什么问题？它和 Supervisor 路由的区别在哪？What problem does the Planner-Executor-Critic structure solve, and how is it different from Supervisor routing?

  国内高频海外高频基础#multi-agent#orchestration#architecture

  分析过程 · 先想清楚再作答

  1. 题眼在后半句。只答「拆解、执行、评审」是在背名词，面试官想确认的是你能不能用图的形状把两种模式分开，而不是靠记忆背模式表。
  2. 先用形状拆：Supervisor 是一个岔路口，运行时在几条路里选一条走，一次只交给一个人，图上只有分叉；Planner-Executor-Critic 是先扇出、再汇合、中间还有一条回边。分叉解决「交给谁」，扇出解决「一件事要拆成几件」，回边解决「谁来验收」。
  3. 再给适用判据：一次只需要一个专家、难点在判断该找谁，用 Supervisor；一件事必须拆成几件且几件之间没有先后依赖，才值得扇出；产出的对错有明确判据、且错了重做比错了发出去便宜，才值得加 Critic。三条判据都不命中就别上这套结构。
  4. 结论要落到代价，这是区分「读过文档」和「上线过」的地方：拆出三件事意味着模型调用次数从一次变成七次起步（拆解一次、三次执行、三次评审），有一轮打回就是九次；延迟被最慢的那件事决定而不是平均值，而且并行只省延迟不省钱。
  5. 可以预期的追问：Critic 一定要单独一个节点吗？答案是不一定——如果验收判据是可以用代码判的（比如 JSON schema 校验、必填字段检查），就别花一次模型调用，代码判更快更准也更便宜。只有判据本身需要理解语义时，Critic 才值得是一次模型调用。

  How to reason about it · think before answering

  1. The hinge is the second half. Reciting plan, execute, review is naming shapes from memory; the interviewer wants to see you separate the two patterns by graph shape.
  2. Separate by shape: a Supervisor is a fork — at runtime it picks one of several paths and hands the work to exactly one agent, so the graph only branches. Planner-Executor-Critic fans out, joins, and adds a back edge. Branching answers who takes this, fan-out answers this must be split into several pieces, the back edge answers who signs it off.
  3. Then give the criteria: use a Supervisor when only one specialist is needed per request and the hard part is picking them; only fan out when a request genuinely splits into independent pieces with no ordering between them; only add a Critic when correctness has an explicit rubric and redoing is cheaper than shipping something wrong. If none of these hold, do not build this.
  4. Land on cost, which is where shipped-it separates from read-the-docs: three subtasks turn one model call into seven (one plan, three executions, three reviews) and nine after a single rejection round; latency is set by the slowest branch rather than the average, and parallelism buys latency, never money.
  5. Expect: does the Critic have to be its own node? Not necessarily — if the rubric is checkable in code (schema validation, required fields), check it in code: faster, cheaper, and more reliable. A Critic earns a model call only when the rubric requires understanding meaning.

  答题要点

  • Supervisor 是分叉（一次派一个人），Planner-Executor-Critic 是扇出加汇合加回边（拆成几件并行做，做完有人验收）
  • 三条适用判据：一次只需一个专家用路由；能拆成互不依赖的几件才扇出；对错有明确判据且重做便宜才加评审
  • 拆解的代价是模型调用从一次涨到七到九次、延迟由最慢的分支决定，而并行只省延迟不省成本
  • 评审判据能用代码判就别用模型判，Critic 只在需要理解语义时才值一次模型调用

  Key points

  • A Supervisor branches (one agent per request); Planner-Executor-Critic fans out, joins, and loops back (split, run in parallel, then sign off)
  • Three criteria: route when one specialist suffices; fan out only for genuinely independent pieces; add review only when the rubric is explicit and redoing beats shipping wrong
  • The cost is seven to nine model calls instead of one, with latency set by the slowest branch — parallelism buys latency, not money
  • If the rubric is checkable in code, check it in code; a Critic deserves a model call only when semantics must be understood
• 多个子任务并行执行、都要写同一份共享状态时，怎么设计才不会互相覆盖？When several subtasks run in parallel and all write the same shared state, how do you design it so they do not clobber each other?

  国内高频海外高频深入#multi-agent#state-management#concurrency

  分析过程 · 先想清楚再作答

  1. 这题的区分度极高，因为大多数人会答成「加锁」或者「用不可变数据结构」——都是从多线程经验迁移过来的答案，但图的执行模型里根本没有并发写内存这回事，写入是被收集起来统一合并的。答错方向比答不全更致命。
  2. 先把机制说对：并行节点各自返回一份增量，框架把同一轮里所有增量按字段收集，再逐字段调用这个字段的合并规则（reducer）算出新值。所以问题不是「怎么加锁」，而是**这个字段的合并规则写得对不对**。
  3. 然后给一条可复用的判断链：先问这个字段同一轮会被几个人写；再问他们写的是不是同一条记录。只有一个写者，默认的后写覆盖就够；多个写者写不同记录，数组追加就够；多个写者写同一条记录的同一份数据，要按主键原地更新；多个写者写同一条记录的不同字段，要做字段级合并。四种情况四种 reducer，这条链能直接迁移到任何框架。
  4. 结论落在最容易踩的那一格：把「更新一条记录」写成「往数组里追加一条同 id 的新版本」。它的症状不是报错，是同一个 id 在状态里有两份、而且哪份在前取决于谁先跑完——下游任何按 id 查的地方都可能拿到过期版本，日志干净、结果偶尔错。
  5. 补一句权衡：也可以不动 reducer，改成每处读状态前先按 id 去重。但读取点有三四处，漏一处就是一个偶发脏读；reducer 只写一次，之后所有读取点自动干净。在字段上解决一次，还是在每个读取点解决 N 次，这是同一个问题的两种成本。
  6. 可以预期的追问：那顺序不确定要不要紧？答：合并规则最好对顺序不敏感（可交换），做不到就必须保证每条记录只有一个写者。本课的按 id 原地更新属于后者——它是最后写入者获胜，靠「一轮里一条记录只有一个执行者」这个前提才安全。

  How to reason about it · think before answering

  1. This one separates people fast, because most candidates answer locks or immutable data structures — instincts carried over from threads. A graph runtime has no concurrent memory writes at all: updates are collected and merged. Answering in the wrong frame is worse than answering incompletely.
  2. Get the mechanism right first: parallel nodes each return a delta, the runtime groups all deltas from the same step by field, then calls that field's reducer to compute the new value. So the question is not how to lock, it is whether that field's reducer is correct.
  3. Then give a reusable chain: how many writers touch this field in one step, and do they write the same record? One writer — last-write-wins is fine. Several writers on different records — appending to a list is fine. Several writers on the same record — upsert by key. Several writers on different fields of the same record — merge per field. Four cases, four reducers, and the chain transfers to any framework.
  4. Land on the common mistake: implementing update this record as append a new version with the same id. The symptom is not an error — the same id exists twice and which one comes first depends on who finished first, so any lookup by id may return the stale version. Clean logs, occasionally wrong results.
  5. Add the trade-off: you can leave the reducer alone and dedupe by id at every read instead. But there are three or four read sites, and missing one is an intermittent stale read; a reducer is written once and every read is clean afterwards. Solve it once on the field, or N times at the read sites.
  6. Expect: does nondeterministic ordering matter? Ideally the reducer is order-insensitive (commutative); if it is not, you must guarantee one writer per record. Upsert-by-id is the latter — it is last-write-wins and is safe only because each record has exactly one executor per round.

  答题要点

  • 图的执行模型里没有并发写内存：节点各返回增量，框架按字段收集后调用该字段的 reducer 合并，所以问题是 reducer 写得对不对，不是加不加锁
  • 判断链：同一轮几个写者、写的是不是同一条记录——单写者用覆盖、多写者写不同记录用追加、多写者写同一条记录用按主键原地更新、写同一条记录的不同字段要字段级合并
  • 最常见的错是把「更新」写成「追加同 id 的新版本」，症状是同 id 两份、顺序取决于谁先跑完、按 id 查会拿到过期版本，而且全程不报错
  • 另一条路是每处读取前手动去重，但读取点有好几处，漏一处就是偶发脏读；reducer 只写一次就一劳永逸
  • 合并规则最好对顺序不敏感；做不到就必须保证一轮里一条记录只有一个写者

  Key points

  • A graph runtime has no concurrent memory writes: nodes return deltas, the runtime groups them per field and calls that field's reducer — so the answer is a correct reducer, not a lock
  • Decision chain: how many writers per step, and same record or not — overwrite, append, upsert by key, or per-field merge
  • The classic bug is implementing update as append-a-new-version-with-the-same-id: two entries per id, order depends on who finished first, lookups return stale data, and nothing ever errors
  • The alternative is deduping at every read site, but there are several and missing one gives an intermittent stale read; a reducer is written once
  • Prefer an order-insensitive reducer; if it is not, guarantee exactly one writer per record per step
• 为什么要给每个子任务设 toolBudget 这样的预算？超了预算之后你会怎么处理？Why give each subtask a tool-call budget, and what do you do when it runs out?

  国内高频海外高频进阶#cost-control#reliability#agent-design

  分析过程 · 先想清楚再作答

  1. 这题的题眼在后半句。前半句几乎人人会答「防止成本失控」，真正拉开差距的是超限之后的动作——答「抛异常」的人基本没做过面向用户的 Agent。
  2. 先把「为什么」说具体。子任务卡住的典型形态不是报错，而是反复查、反复不满意、再查——模型不会喊累，它会把额度花光为止。整轮对话的成本封顶是外层的闸，子任务预算是内层的闸；粒度细到单件事的好处是超支时你能精确指出是哪一件失控了，而不是只看到这次对话贵了。
  3. 再点一个容易被忽略的设计点：预算必须是子任务级的，不是单次执行级的。有评审回路时，被打回重做也得计费，否则打回两次实际额度就翻三倍，这道闸等于没设。
  4. 结论是超限的处理：降级返回已有结果并打上标记，不抛错。理由要说透——抛错等于把「这件事只做了一半」升级成「整个请求失败」，用户等了几秒最后看到一句服务异常，可他其实只是没拿到三件事里的一件。给出两件半的答案并说明哪半件没做成，永远比一个错误页有用。
  5. 降级标记本身也要说：它让降级变成可观测、可统计的事实，而不是日志里的一句话。上层据此决定要不要转人工，监控据此画降级率——两个平均分一样的系统，降级率百分之三十和百分之三完全不是一回事。
  6. 可以预期的追问：预算该设多少？答案是从「这件事正常需要几次工具调用」反推再留一点余量，不是拍脑袋取整数；同时要有第二个维度的闸（挂钟时间或 token 数），因为一次超长的工具调用同样能拖垮请求，而它只算一次。

  How to reason about it · think before answering

  1. The hinge is the second half. Everyone can say it controls cost; what separates people is what happens when the budget runs out. Answering throw an exception usually means you have never shipped a user-facing agent.
  2. Make the why concrete: a stuck subtask rarely errors — it queries, dislikes the result, and queries again. The model never gets tired; it will spend whatever you allow. A per-conversation cap is the outer gate, a per-subtask budget is the inner one, and the finer grain tells you which piece went out of control instead of only that the conversation was expensive.
  3. Add the design point people miss: the budget must be per subtask, not per execution. With a review loop, retries have to draw on the same budget, or two rejections triple the real allowance and the gate is meaningless.
  4. The conclusion is the exhaustion path: degrade — return what you already have with a flag — rather than throw. Explain why: throwing upgrades this piece is half done into the whole request failed. The user waited several seconds and gets an error page, when in reality only one of three pieces is missing. Two and a half answers plus a clear note beats an error page every time.
  5. Say something about the flag too: it turns degradation into an observable, countable fact instead of a log line. The layer above decides whether to escalate to a human, and monitoring plots a degradation rate — two systems with the same average score but 30 percent versus 3 percent degradation are not the same system.
  6. Expect: how big should the budget be? Derive it from how many tool calls the task normally needs plus margin, not a round number pulled from the air. And pair it with a second dimension — wall-clock or tokens — because one very slow tool call can ruin a request while counting as a single call.

  答题要点

  • 子任务卡住的典型形态是反复查而不是报错，模型会把额度花光为止；整轮封顶是外层闸，子任务预算是内层闸，细粒度让你能定位到是哪一件失控
  • 预算必须是子任务级而不是单次执行级，否则被评审打回两次实际额度就翻三倍
  • 超限必须降级返回已有结果并标记，不能抛错——抛错把「做了一半」升级成「整个请求失败」，用户连已经查到的部分都拿不到
  • 降级标记让降级率变成可统计指标，上层据此决定转人工，评估据此区分两个平均分相同的系统
  • 预算大小从这件事正常需要几次工具调用反推并留余量，同时配一个时间或 token 维度的闸

  Key points

  • A stuck subtask loops rather than errors, and the model will spend whatever you allow; a conversation cap is the outer gate, a subtask budget the inner one that localises the blowup
  • The budget must be per subtask, not per execution, or two review rejections triple the real allowance
  • On exhaustion, degrade and flag rather than throw — throwing upgrades half done into whole request failed and discards what was already retrieved
  • The degradation flag makes the degradation rate a real metric for escalation and evaluation
  • Size the budget from the task's normal tool-call count plus margin, and pair it with a wall-clock or token gate
• Critic 的评审回路怎么防止陷入死循环？除了次数上限还有什么要防的？How do you keep a Critic review loop from spinning forever, and what else needs guarding besides a retry cap?

  国内高频海外高频进阶#reflection#loop-guard#reliability

  分析过程 · 先想清楚再作答

  1. 问「除了次数上限还有什么」，说明面试官已经预设你会答上限，真正在考的是你有没有真的跑过这条回路。只答上限的人拿基础分，能说出另外两种失效方式的才算过。
  2. 第一种就是无限打回：每改一版评审者挑一个新毛病，永远收敛不了。上限的作用不是省钱，是**保证流程一定会结束**。本课取最多打回 2 次、共 3 次执行，这个量级的取法是「一次有效的修改通常在第二次就完成，第三次还不行说明判据本身有问题」。
  3. 第二种是打回不说人话：评审者只回一句「不合格」，执行者拿不到可执行信息，第二稿原样再交一遍，于是必然打满上限、白烧三倍的钱。所以打回必须带具体理由，而且理由要回写进子任务的目标里带给执行者——「缺了退款结论，请补上」才是可执行的，「质量不佳」不是。
  4. 第三种最危险也最少被提到：评审者和执行者用同一个模型、同一套提示词时，它倾向于认可自己的输出。同一个模型对「什么算好答案」的偏好是一致的，让它复核自己刚写的东西，通过率会高得离谱，这道工序等于没有。缓解手段按性价比排：给评审者一份可核对的客观验收要求；换一个不同的模型来评审，哪怕更便宜；把评审做成逐条打分而不是一句结论。
  5. 还要点一句框架的兜底与业务上限的区别：编排框架通常自带一个递归步数上限，但那是最后一道保险丝，不能当业务上限用——它是全图的，你不知道是哪条回路失控；而且它触发时抛异常，你连已有结果都拿不到，正好违背「降级返回」的原则。
  6. 可以预期的追问：上限用完了返回什么？答：返回已有结果并标记降级，同时把最后一次的评审意见一起带出去，让上层能判断要不要转人工——这条回路的价值不只是修好，还包括「修不好时说清楚差在哪」。

  How to reason about it · think before answering

  1. Asking what else besides a cap tells you the interviewer already expects the cap. What is really being tested is whether you have run this loop for real. The cap earns baseline credit; naming the other two failure modes is what passes.
  2. Failure one is infinite rejection: every revision draws a new complaint and nothing converges. The cap exists to guarantee termination, not to save money. Two rejections and three executions is a reasonable default, because an effective fix usually lands on the second attempt — if the third still fails, the rubric itself is the problem.
  3. Failure two is a rejection with no actionable content. If the reviewer only says not good enough, the executor has nothing to act on and resubmits the same thing, burning the full cap. Rejections must carry a specific reason, and that reason must be written back into the subtask goal. Missing the refund conclusion, please add it is actionable; poor quality is not.
  4. Failure three is the dangerous one people rarely mention: when reviewer and executor share a model and a prompt, the reviewer tends to approve its own output. A single model has consistent preferences about what a good answer looks like, so pass rates go implausibly high and the review step becomes theatre. Mitigations by value: give the reviewer an objective, checkable rubric; use a different model even a cheaper one; score item by item rather than emitting one verdict.
  5. Also distinguish the framework's safety net from your business cap: orchestration frameworks usually ship a recursion limit, but that is a last-resort fuse — it is graph-wide so you cannot tell which loop ran away, and it throws, which means you lose the partial results you were supposed to degrade to.
  6. Expect: what do you return once the cap is used up? Return what you have, flag it as degraded, and carry the last review comment out with it so the layer above can decide whether to escalate. The loop's value is not only fixing things — it is stating precisely what could not be fixed.

  答题要点

  • 次数上限的作用是保证流程一定会结束，不是省钱；本课取最多打回 2 次、共 3 次执行
  • 打回必须带具体、可执行的理由并回写进子任务目标，只说「不合格」会让执行者原样重交、必然打满上限
  • 最危险的是评审者与执行者同模型同提示词，它倾向于认可自己的输出，通过率虚高、这道工序等于没有
  • 缓解手段：给客观可核对的验收要求、换一个模型来评审、逐条打分而不是一句结论
  • 框架自带的递归上限只是保险丝，不能当业务上限：它是全图的、触发时抛异常，连已有结果都拿不到
  • 上限用完要返回已有结果加降级标记，并把最后一次评审意见带出去，供上层决定是否转人工

  Key points

  • A retry cap exists to guarantee termination, not to save money — two rejections, three executions total
  • Rejections must carry specific, actionable reasons written back into the subtask goal; not good enough guarantees an identical resubmission and a maxed-out cap
  • The most dangerous failure is a reviewer sharing model and prompt with the executor: it approves its own output, pass rates inflate, and the step becomes theatre
  • Mitigate with an objective checkable rubric, a different model for review, and item-by-item scoring instead of a single verdict
  • The framework's recursion limit is a fuse, not a business cap: it is graph-wide and it throws, so you lose the partial results you meant to degrade to
  • When the cap is spent, return what you have with a degraded flag plus the last review comment so the layer above can escalate

D18 历史保真与摘要、多模态占位、checkpointer 持久化

• 记忆应该怎么分层？短期上下文、摘要、长期记忆分别放什么、丢了会怎么样？How should agent memory be layered? What belongs in short-term context, in summaries, and in long-term memory — and what happens when each is lost?

  国内高频海外高频基础#memory#context-management#multi-agent

  分析过程 · 先想清楚再作答

  1. 这题的区分度不在能不能列出三层，而在能不能说出**每一层丢了会怎样**。只报名词的答案，面试官听不出你有没有真的运维过。
  2. 先给一条可复用的拆法：按「谁在读它、活多久、丢了能不能补」三个问题去分，任何一个记忆方案都能被这三问切开。
  3. 短期上下文是这一次请求要发给模型的那个消息数组，随请求结束作废，全量进 token 账单；它丢了只影响这一轮的连贯性，原文还在你自己的会话记录里，可以重放。
  4. 摘要是短期上下文的派生数据，用来在窗口顶到之前把早期内容压短；它丢了可以重新生成——**前提是原文另存了一份**。所以摘要绝不能覆盖原文，这是「压缩不可逆」那条纪律的实际落点。
  5. 长期记忆是跨会话的用户事实与偏好，不进消息数组，存在外部检索层里按需捞几条注入；它丢了的表现是「这个用户被系统忘光了」，不影响单次可用，但产品价值直接掉一层。
  6. 多 Agent 还要补第四层，也是最容易被忽略的一层：**图的执行状态**。它包含消息、共享工作区、评审轮次、降级标记，是唯一一份会被检查点持久化并在恢复时重放的数据。它丢了的后果最重——一次已经花掉九次模型调用的执行必须从头再来，而且用户界面还停在转圈。
  7. 可以预期的追问：摘要该放在消息数组里还是单独一个字段？答单独字段，理由是原文与派生数据要分开存，才可能换一种策略重新生成；混在一起之后你分不清哪条是真发生过的、哪条是事后编的。

  How to reason about it · think before answering

  1. The discriminator here is not listing three layers, it is saying what breaks when each one is lost. An answer that only names the layers tells the interviewer you have never operated one.
  2. Offer a reusable split first: sort any memory scheme by who reads it, how long it lives, and whether it can be rebuilt after loss. Those three questions cut through every design.
  3. Short-term context is the message array sent to the model this turn. It dies with the request and is billed in full every turn. Losing it only costs coherence for that turn, because the raw transcript still lives in your own store and can be replayed.
  4. A summary is derived from short-term context, produced to shrink early turns before the window fills. It can be regenerated after loss — but only if the raw transcript was stored separately. That is the practical reason a summary must never overwrite the original.
  5. Long-term memory holds cross-session user facts and preferences. It never enters the message array; it lives in a retrieval layer and a few hits get injected on demand. Losing it means the system forgot the user — single requests still work, but the product gets noticeably worse.
  6. Multi-agent adds a fourth layer people usually miss: graph execution state — messages, shared workspace, review rounds, degraded flags. It is the only copy that gets checkpointed and replayed on resume, and losing it is the most expensive failure: a run that already burned nine model calls starts over while the user watches a spinner.
  7. Expect the follow-up: should the summary live inside the message array or in its own field? Say its own field — keeping raw and derived data apart is what lets you regenerate with a different strategy later; merged together you can no longer tell what actually happened from what was written after the fact.

  答题要点

  • 按「谁在读、活多久、丢了能不能补」三问分层，比背名词有用
  • 短期上下文：本轮请求的消息数组，随请求作废，全量计费，丢了可从原始记录重放
  • 摘要：短期上下文的派生数据，可重新生成，前提是原文另存——所以摘要不能覆盖原文
  • 长期记忆：跨会话的用户事实，存在检索层按需注入，丢了是「系统忘了这个人」
  • 多 Agent 多一层图执行状态：消息 + 工作区 + 评审轮次 + 降级标记，会被检查点持久化并在恢复时重放，丢了最贵
  • 摘要放独立字段而不是塞回消息数组，原文与派生数据分开存才可能换策略重生成

  Key points

  • Layer by who reads it, how long it lives, and whether it can be rebuilt — that beats reciting names
  • Short-term context: this turn's message array, discarded after the request, billed in full, replayable from your own transcript
  • Summary: derived from short-term context and regenerable, but only if the raw transcript is stored separately — so it must never overwrite the original
  • Long-term memory: cross-session user facts in a retrieval layer, injected on demand; losing it means the system forgot the user
  • Multi-agent adds graph execution state — messages, workspace, review rounds, degraded flags — checkpointed and replayed on resume, and the most expensive to lose
  • Keep the summary in its own field rather than back in the message array, so raw and derived data stay separable
• 长对话做摘要时，怎么保证关键信息不丢？在多 Agent 场景下这件事有什么特别的？When summarizing a long conversation, how do you keep the critical information from being lost — and what is different about this in a multi-agent system?

  国内高频海外高频进阶#context-compression#multi-agent#reliability

  分析过程 · 先想清楚再作答

  1. 题眼在后半句。只答「保留用户约束、保留最近几轮」是单 Agent 的标准答案，能过但不出彩；面试官问「多 Agent 有什么特别的」，是在看你有没有真的在协作图里踩过这个坑。
  2. 先把单 Agent 那半答扎实：触发用阈值不用定时器，本课口径是消息超过 20 条或估算超过 8000 token（一个字符算一个 token，故意高估，低估会让阈值永远触发不了）；保留最近 6 条原文不动；切口必须对齐到一轮的开头，切在工具调用与工具结果之间会让下一次请求出现悬空消息，多数厂商直接返回 400。
  3. 然后给出多 Agent 那半的关键差别：单 Agent 里摘要丢的是**细节**，多 Agent 里摘要丢的是**判据**。评审者判一份产出合不合格，靠的是分清「这句是待验收的产出、那句是验收要求」；一段把发言人抹平的流水摘要读起来通顺，但评审拿它做不了任何判断。
  4. 所以多 Agent 的摘要有一条额外硬要求：每条被压掉的消息，在摘要里都要留下「第几条 + 谁说的」这两个坐标。实现只有一行——把转录写成带序号和角色前缀的形式再交给模型。
  5. 再补一条边界，这条最能显出你写过：**摘要只对自然语言历史动手，不碰任何结构化字段**。把共享工作区压成一句话，「按 id 找到某条子任务、比对验收要求」就整个失效了，结构化数据压成自然语言就再也回不去。附件字段更是碰不得——它存的是引用不是内容，摘要掉等于把那个对象变成孤儿。
  6. 可以预期的追问：摘要用哪个模型、失败了怎么办？答可以用更便宜的小模型（它只做归纳不做推理），失败时的正确行为是**跳过这一轮压缩继续跑**并告警，而不是让整次执行失败——阈值定在七八成就是为了留出这次抢救余量。

  How to reason about it · think before answering

  1. The hinge is the second half. Answering only keep user constraints and the last few turns is the standard single-agent answer — passable, not memorable. Asking what is different in multi-agent is asking whether you have actually hit this in a collaboration graph.
  2. Get the single-agent half solid first: trigger on thresholds, never a timer. This course uses more than 20 messages or an estimated 8000 tokens, counting one character as one token — deliberately high, because underestimating means the threshold never fires. Keep the last 6 messages verbatim. Align the cut to a turn boundary: cutting between a tool call and its result produces a dangling message and most vendors return 400.
  3. Then name the real difference: in a single agent a summary loses detail; in a multi-agent graph a summary loses the criteria. A critic decides whether output passes by telling apart what is being reviewed from what the requirement was. A smooth narrative summary that flattens speakers reads fine and is useless to the critic.
  4. So multi-agent summarization has one extra hard requirement: every compressed message must leave behind two coordinates — its index and its speaker. The implementation is one line: build the transcript with numbered, role-prefixed entries before handing it to the model.
  5. Add the boundary that shows you have shipped this: summarize natural-language history only, never structured fields. Compressing the shared workspace into a sentence kills every lookup by task id and every comparison against an acceptance requirement, and structured data does not come back. Attachments are even more off-limits — they hold a reference, not content, so summarizing one orphans the underlying object.
  6. Expect the follow-up: which model writes the summary, and what if it fails? A cheaper small model is fine since the job is condensation, not reasoning. On failure the correct behaviour is to skip this round of compression, keep running, and alert — not to fail the whole execution. Setting the threshold at seventy or eighty percent exists precisely to leave that rescue room.

  答题要点

  • 触发用阈值不用定时器：超过 20 条或估算超过 8000 token，token 按一字符一 token 保守高估
  • 保留最近 6 条原文不动，切口必须对齐到一轮开头，否则会出现有调用没结果的悬空消息、请求直接 400
  • 多 Agent 的差别：摘要丢的不是细节而是判据，评审者靠「谁在第几步说的」区分产出与验收要求
  • 所以每条被压掉的消息都要在摘要里留下条号与发言人，实现就是把转录写成带序号和角色的形式
  • 只压自然语言历史，不碰共享工作区这类结构化字段，更不能碰存引用的附件字段
  • 摘要可用更便宜的小模型；摘要调用失败时跳过这一轮压缩并告警，不要让整次执行失败

  Key points

  • Trigger on thresholds, not timers: more than 20 messages or an estimated 8000 tokens, counting one character as one token to stay conservative
  • Keep the last 6 messages verbatim and align the cut to a turn boundary, or you ship a dangling tool call and the request 400s
  • The multi-agent difference: a summary loses criteria, not just detail — the critic needs to know who said what and at which step
  • So every compressed message keeps its index and speaker in the summary; the implementation is a numbered, role-prefixed transcript
  • Summarize natural-language history only — never the shared workspace or other structured fields, and never the attachment references
  • A cheaper small model is fine for summarizing; if the call fails, skip compression for this round and alert rather than failing the run
• 从 checkpoint 恢复执行（replay）需要注意什么？说几个真实会踩的坑。What do you need to watch out for when replaying execution from a checkpoint? Give failure modes you would actually hit.

  国内高频海外高频深入#checkpointing#replay#reliability

  分析过程 · 先想清楚再作答

  1. 这题最容易答成「读出来接着跑就行」。区分度在于你能不能说出**这些坑几乎全是静默的**——不抛异常、日志干净、结果看起来也对，只有对比数据时才发现不对。能说出这一点，答案就已经赢了一半。
  2. 先给一条推导链：检查点里存的是「当时那个版本的代码眼里的状态形状」，恢复就是把它塞回今天这个版本的代码里。所以所有坑都来自**两端不一致**：数据的形状、执行的入口、和那些不该被重放的东西。
  3. 坑一，恢复时又把输入喂了一遍。恢复的入口是不带输入地调用，状态已经在检查点里；带着原来那句话再调一次，框架会把它当成一次新的状态更新叠在中断点上，历史变成两份。它不报错。
  4. 坑二，分叉忘了带检查点 id。只给会话 id 拿到的是这条线最新的状态，于是「从第 2 步重来」变成了「在最后一步后面接着写」。同样不报错，只有对比子任务列表才看得出来。
  5. 坑三，版本兼容。改一个字段名、加一个必填字段，库里的老检查点就和新代码对不上；而缺字段读出来是 undefined，拼进文案就是字符串「undefined」，参与算术就是 NaN——比如工具预算的上限判断，一旦变成 NaN 比较，恒为假，预算上限在恢复出来的那条线上彻底失效。正确做法是在读的那一侧迁移，迁移函数只补默认值和改名、不做业务判断，绝不能失败。
  6. 坑四，不该被重放的东西进了状态。一次性的人工干预（比如人工改派）如果写进图状态，就会被检查点持久化并在每次恢复时重放一遍。判断口径：这条信息说的是「这一次执行怎么跑」还是「这个会话是什么」，前者进运行时配置，后者才进状态。
  7. 可以预期的追问：待执行的并行子任务存不存？答存——检查点里除了状态快照还有一份「还没跑的那几步，连参数一起」，所以恢复不用重跑规划节点；但它存在框架的内部通道里，自研存储层只实现「存状态」而漏掉这一半，恢复出来的图会看起来跑完了、其实一件活都没派出去。

  How to reason about it · think before answering

  1. The easy failure is answering just load it and keep going. The discriminator is recognising that almost every replay bug is silent — no exception, clean logs, plausible output, and you only notice when you diff the data. Saying that up front wins half the question.
  2. Give a chain first: a checkpoint stores the state shape as the code of that moment understood it, and replay pushes it back into today's code. So every failure comes from a mismatch across those two ends — the shape of the data, the entry point of execution, and things that should never have been replayed at all.
  3. Trap one: feeding the input again on resume. Resume takes no input; the state is already in the checkpoint. Passing the original message once more makes the framework treat it as a fresh update stacked on the interrupt point, and the history quietly doubles. Nothing throws.
  4. Trap two: forking without a checkpoint id. With only the thread id you get that thread's latest state, so start over from step 2 silently becomes append after the last step. Again nothing throws; you only see it by diffing the task list.
  5. Trap three: version drift. Rename a field or add a required one and every old checkpoint stops matching the new code. A missing field reads as undefined, which renders as the literal string undefined in user-facing text and as NaN in arithmetic — a tool-budget ceiling compared against NaN is always false, so the budget silently stops existing on resumed threads. Migrate on read, and keep the migration to defaults and renames only: it must never fail.
  6. Trap four: replayable data that should not be replayed. A one-off human override written into graph state gets checkpointed and re-applied on every resume. The test: does this describe how this run executes, or what this conversation is? The former belongs in runtime config, only the latter in state.
  7. Expect the follow-up: are pending parallel tasks preserved? Yes — a checkpoint holds not just the state snapshot but the steps not yet run, arguments included, so the planner does not re-run. But they live in a framework-internal channel, so a hand-rolled store that persists state and forgets that half will resume into a graph that looks finished while no work was ever dispatched.

  答题要点

  • 先点破共性：replay 的坑几乎全是静默的，不报错、日志干净、结果看着也对
  • 恢复不要带输入，带了就是在中断点上又追加一次，历史变成两份
  • 分叉必须带检查点 id，只给会话 id 会落在最新状态上，「从第 2 步重来」变成「接着往后写」
  • 版本兼容：缺字段读出来是 undefined 或 NaN，会让预算上限之类的比较恒为假；在读的那一侧迁移，迁移只补默认值和改名且不能失败
  • 一次性的人工干预不要进图状态，否则会被持久化并在每次恢复时重放；「这次怎么跑」进配置，「这个会话是什么」才进状态
  • 待执行的并行子任务连参数一起存在检查点里，所以恢复不重跑规划；自研存储层漏掉这一半，恢复出来的图会一件活都不派

  Key points

  • Lead with the pattern: replay bugs are almost all silent — no exception, clean logs, plausible output
  • Resume takes no input; passing one appends another update at the interrupt point and doubles the history
  • Forking requires the checkpoint id — thread id alone lands on the latest state, turning start over from step 2 into append after the end
  • Version drift: missing fields read as undefined or NaN, so comparisons like a tool-budget ceiling become permanently false. Migrate on read, restricted to defaults and renames, and never let it fail
  • Keep one-off human overrides out of graph state or they get persisted and re-applied on every resume — how this run executes belongs in config, what this conversation is belongs in state
  • Pending parallel tasks are stored with their arguments, so the planner does not re-run; a hand-rolled store that skips that half resumes into a graph that dispatches nothing
• checkpointer 在多 Agent 系统里解决了什么问题？它的代价是什么？What problem does a checkpointer solve in a multi-agent system, and what does it cost?

  国内高频海外高频进阶#checkpointing#cost#operations

  分析过程 · 先想清楚再作答

  1. 这题的下半句才是考点。只答「能恢复、能容错」是功能介绍，任何文档都写着；面试官想听的是你有没有算过这笔账，以及知不知道它会在哪里疼。
  2. 先把价值说具体，用钱和时间说：一次带评审回路的多 Agent 执行要调九次模型，跑到第七次进程被换版本重启，没有检查点就是九次全废、用户界面还停在转圈。有检查点则从上一个签字点接着跑，已经跑完的节点一次都不重跑——它买的是「失败的粒度从一整次执行降到一个节点」。
  3. 顺带说清它解锁的另外三件事，这三样单靠重试做不到：**人工审批闸口**（在某个节点前停下等人点确认，状态就停在那儿）、**时间旅行调试**（回到出问题那一步之前看状态长什么样）、**分叉对比**（从同一个检查点跑两种走法，比较结果，这也是评估的基础设施）。
  4. 然后是代价，三笔要说全。第一笔，**状态越大写得越慢**，而且是每一步都写一份——一次请求写六个检查点，状态里多一个字节就要多写六遍。所以附件存引用不存内容，检索结果存文档 id 不存全文。
  5. 第二笔，**存储本身有上限**。用 jsonb 存的话，硬上限很远，但单行超过大约两 KB 就会被挪到外存、每次读写多一次 IO，所以真正的工程线是「别让单个检查点变成几百 KB」，而不是那个理论上限。
  6. 第三笔也是最容易被忽略的：**版本兼容**。检查点是长期存活的数据，你每改一次状态形状就欠下一笔迁移债，而缺字段读出来通常不报错，只是静默给出 undefined 或 NaN。这一条决定了状态字段要尽早占好位子——图状态的形状一旦被持久化，改字段就不是改代码，是数据迁移。
  7. 可以预期的追问：那检查点要不要清理？答要，按会话线设保留期与归档策略，否则这张表会随日活线性膨胀；另外要留意它是敏感数据——图状态里有完整对话，删除用户数据时这张表必须一起处理。

  How to reason about it · think before answering

  1. The second half is the real question. Answering it enables recovery and fault tolerance is a feature blurb any doc carries. The interviewer wants to know whether you have done the arithmetic and where it hurts.
  2. Make the value concrete in money and time: one multi-agent run with a review loop costs nine model calls. If the process is restarted for a deploy at call seven, without checkpoints all nine are wasted and the user is still watching a spinner. With them the run continues from the last signed-off point and no completed node re-runs. What you bought is a smaller unit of failure — a node instead of a whole run.
  3. Mention the three things it unlocks that retries alone cannot: human approval gates (pause before a node and the state simply waits), time-travel debugging (go back to just before the bad step and inspect state), and forking for comparison (run two variants from one checkpoint) — which is also the infrastructure evaluation is built on.
  4. Then the costs, all three. First, bigger state means slower writes, and it is written at every step: one request produces six checkpoints, so a byte added to state is six bytes written. Hence attachments hold references, not content, and retrieval results hold document ids, not full text.
  5. Second, the store has limits. With jsonb the hard cap is far away, but a row past roughly two kilobytes gets pushed to out-of-line storage and costs an extra IO on every read and write. The real engineering line is do not let a single checkpoint reach hundreds of kilobytes, not the theoretical cap.
  6. Third, the one people forget: version compatibility. Checkpoints are long-lived data, so every change to the state shape incurs migration debt, and missing fields usually do not throw — they silently yield undefined or NaN. This is why state fields should be reserved early: once a shape is persisted, changing a field is a data migration, not a code edit.
  7. Expect the follow-up: do checkpoints need cleanup? Yes — retention and archival per thread, or the table grows linearly with active users. Also treat it as sensitive data: graph state contains full conversations, so it must be included whenever you delete a user's data.

  答题要点

  • 核心价值：把失败的粒度从「一整次执行」降到「一个节点」，九次模型调用的执行不会因为一次重启全废
  • 还解锁三件重试做不到的事：人工审批闸口、时间旅行调试、从同一个检查点分叉对比（也是评估的基础设施）
  • 代价一，状态越大写得越慢，而且每一步都写一份——所以附件存引用、检索结果存文档 id
  • 代价二，存储有上限：jsonb 单行超过约两 KB 就外存、多一次 IO，工程线是别让单个检查点到几百 KB
  • 代价三，版本兼容：状态形状改一次就欠一笔迁移债，缺字段静默给出 undefined 或 NaN；所以字段要尽早占位
  • 运维上还要有保留期与归档，并把它当敏感数据处理——图状态里有完整对话，删用户数据时必须一起删

  Key points

  • Core value: it shrinks the unit of failure from a whole run to a single node, so a nine-call run is not wasted by one restart
  • It also unlocks three things retries cannot: human approval gates, time-travel debugging, and forking from one checkpoint to compare variants — the substrate evaluation is built on
  • Cost one: bigger state writes slower, and it is written at every step — hence references for attachments and document ids for retrieval results
  • Cost two: storage limits — a jsonb row past roughly two kilobytes goes out-of-line and costs an extra IO, so the practical line is keeping a checkpoint well under hundreds of kilobytes
  • Cost three: version compatibility — every change to the state shape is migration debt, and missing fields silently yield undefined or NaN, which is why fields should be reserved early
  • Operationally you need retention and archival, and you must treat it as sensitive data: graph state holds full conversations and must be purged with the user's data

D19 跨服务 Agent 集成：用户级 JWT 铸造、JWKS 验签、inject/memory/usage 三类接口、幂等 externalId

• 两个服务之间调用，你会用服务级令牌还是用户级令牌？分别适用于什么场景？For service-to-service calls, would you use a service token or a user token? When does each apply?

  国内高频海外高频进阶#auth#security#api-design

  分析过程 · 先想清楚再作答

  1. 题眼在「分别」。答「用户级更安全」就把一道设计题做成了口号题——面试官想看你能不能说出两者各自成立的条件，以及选错的具体代价。
  2. 先给判断依据，一句话就能拆开：这次调用**有没有一个具体的用户在背后**。有，就必须是用户级；没有（拉配置、上报指标、跑对账批处理），服务级才是对的，硬塞一个用户 id 进去反而是伪造审计记录。
  3. 然后把用户级的三条理由说成代价而不是优点：服务级令牌泄露一次等于全量用户数据泄露，用户级泄露一张只丢一个用户且十五分钟自动作废；服务级在审计日志里只能查到「某服务调了一次」，查不到替谁操作；下游做用户级权限判断时，服务级令牌逼着它去信请求体里的 userId，而那是调用方可以随便写的。
  4. 补一句生产视角：两者不是二选一，真实系统里常常是「服务级令牌用来换用户级令牌」——调用方先用自己的服务凭证证明自己是谁，再申请一张代表某个用户的短期令牌。这样服务凭证只出现在铸造这一步，不出现在每一次业务调用里。
  5. 可以预期的追问一：令牌泄露了怎么办？答案要分两层——短有效期（本课 15 分钟）是止损的主力，撤销列表按 jti 拉黑是补充；不要上来就说「用黑名单」，那等于给每次验签加一次数据库查询，把无状态验签的好处全赔进去了。
  6. 可以预期的追问二：那 scope 该切多细？给一条可操作的判据——按「读写不对称的风险」切，读错了泄露信息、写错了污染数据且会持续影响后续每一轮对话，所以 read 和 write 必须分开；再细就要看有没有真实的调用方只需要其中一半。

  How to reason about it · think before answering

  1. The hinge is each. Answering user tokens are safer turns a design question into a slogan — the interviewer wants the conditions under which each one is correct, and the concrete cost of choosing wrong.
  2. Give the deciding question first: is there a specific user behind this call? If yes, it must be a user token. If not — fetching config, reporting metrics, running a reconciliation batch — a service token is the right answer, and stuffing in a user id would fabricate audit history.
  3. Then state the three reasons as costs, not virtues. A leaked service token means every user's data at once; a leaked user token means one user, and it expires in fifteen minutes. Audit logs with a service token only show that some service called, never on whose behalf. And a downstream service doing per-user authorization is forced to trust a userId in the request body, which the caller writes freely.
  4. Add the production view: it is rarely either-or. Real systems use the service credential to obtain user tokens — the caller proves who it is once, then mints a short-lived token representing one user. The service credential then appears only at the minting step, never on every business call.
  5. Expect: what if a token leaks? Answer in two layers — a short lifetime (fifteen minutes here) does most of the containment, and a jti denylist is the supplement. Do not lead with a denylist: it puts a database lookup in front of every verification and gives away the whole point of stateless verification.
  6. Expect: how fine-grained should scopes be? Offer a usable rule — split along asymmetric risk. A bad read leaks information; a bad write poisons data that keeps influencing every later turn. So read and write always split; finer than that only if a real caller genuinely needs just one half.

  答题要点

  • 判断依据是「这次调用背后有没有一个具体用户」：有就用用户级，没有（配置、指标、对账批处理）才用服务级
  • 服务级令牌泄露的爆炸半径是全量用户，用户级只影响一个用户且短期自动失效
  • 审计要能落到人：只有 sub 字段能回答「当时是替谁操作的」
  • 下游要做用户级权限判断时，服务级令牌逼着它去信请求体里的 userId，而那是调用方可以伪造的
  • 生产里常见组合：服务凭证只用来换取代表某个用户的短期令牌，不出现在每次业务调用里
  • 泄露后的止损顺序是短有效期优先、jti 撤销列表补充，别一上来就上黑名单换掉无状态验签

  Key points

  • The deciding question is whether a specific user stands behind the call: yes means user token, no (config, metrics, reconciliation) means service token
  • A leaked service token exposes every user; a leaked user token exposes one and expires on its own
  • Auditing has to reach a person — only the sub claim answers who the call was made on behalf of
  • With a service token the downstream must trust a userId in the request body, which the caller can forge
  • Common production shape: the service credential only buys short-lived per-user tokens and never appears on business calls
  • After a leak, short lifetimes do the containment and a jti denylist supplements it — do not trade away stateless verification by default
• JWKS 验签是怎么工作的？为什么跨服务场景下它比共享密钥更合适？How does JWKS-based verification work, and why does it fit cross-service scenarios better than a shared secret?

  国内高频海外高频基础#auth#jwt#security

  分析过程 · 先想清楚再作答

  1. 这是本章的送分题，但送分题也有区分度：能不能把「密钥轮换」这件事讲成一个具体的运维动作，而不是一句「更方便管理」。
  2. 先讲机制，三句话：签发方持私钥签名，公钥集合挂在一个固定地址上（本课用 /.well-known/jwks.json）；令牌头部带一个 kid，验签方按 kid 从集合里挑对应的公钥；验签只用公钥，所以这个地址是公开的，谁都能拉。
  3. 再讲为什么比共享密钥好，三条都要落到运维动作上：轮换不用两边同时发版（新旧两把公钥并存一段时间，等老令牌自然过期再摘旧的）；验签方拿到的只是验签能力而不是签名能力，被入侵也伪造不出令牌；多一个调用方不用多散一份密钥出去。
  4. 然后主动补上最容易被忽略的一段：验签不等于验完。签名合法只说明「这确实是那个签发方签的」，还必须校验 iss、aud、exp——**漏掉 aud 是跨服务集成里最常见的事故**，因为签发方给别的下游服务签的令牌，签名一样合法，不校验受众就等于替别人的接口开门。
  5. 工程细节可以再加两条：公钥集合要缓存，但遇到没见过的 kid 要能主动重拉，否则轮换那一刻会集体失败；以及时钟偏移，exp 校验要留一点容忍度，但容忍度不能大到把短有效期的意义抵消掉。
  6. 可以预期的追问：那 HS256 是不是就不能用了？答「同一个服务自己签自己验时它没问题，而且更快」——判据是签名方和验签方是不是同一个信任域，跨了域就必须非对称。这么答显得你在做权衡而不是背结论。

  How to reason about it · think before answering

  1. This is the giveaway question of the chapter, but it still separates people: can you turn key rotation into a concrete operational sequence rather than saying it is easier to manage?
  2. Describe the mechanism in three sentences. The issuer holds the private key and signs; the public key set is published at a fixed address (/.well-known/jwks.json here); the token header carries a kid, and the verifier picks the matching public key from the set. Verification needs only public material, so the endpoint is public by design.
  3. Then give three reasons, each as an operational action: rotation needs no synchronized deploy on both sides (publish the new public key, let both coexist, drop the old one after old tokens expire); the verifier holds verification power, not signing power, so compromising it does not let anyone forge tokens; and adding a caller does not scatter another copy of a secret.
  4. Volunteer the part people forget: verifying the signature is not the whole check. A valid signature only proves the issuer signed it. You still validate iss, aud and exp — and missing aud is the most common cross-service incident, because a token the issuer signed for a different downstream is equally well signed, so skipping audience means holding the door open for someone else's API.
  5. Two engineering details worth adding: cache the key set but refetch on an unknown kid, or rotation day becomes a mass failure; and allow a small clock skew on exp, but not so large that it cancels out the point of short lifetimes.
  6. Expect: so is HS256 unusable? Answer that it is fine when one service signs and verifies its own tokens, and it is faster. The criterion is whether signer and verifier sit in the same trust domain; across domains, asymmetric is mandatory. Framing it as a trade-off shows judgment rather than memorization.

  答题要点

  • 机制：私钥签名、公钥集合挂在固定地址、令牌头部带 kid、验签方按 kid 取公钥
  • 轮换不用两边同时发版：新旧公钥并存，等老令牌自然过期再摘旧的
  • 验签方只拿到验签能力而不是签名能力，被入侵也伪造不出令牌
  • 调用方增加不需要多散一份密钥，公钥公开本来就是设计意图
  • 验签之外必须校验 iss、aud、exp，漏掉 aud 等于替别的下游服务开门
  • 缓存公钥集合但要能按未知 kid 主动重拉；HS256 在同一信任域内自签自验仍然是合理选择

  Key points

  • Mechanism: private key signs, public key set sits at a fixed URL, the token header carries a kid, the verifier selects by kid
  • Rotation needs no synchronized deploy: publish the new key, let both coexist, retire the old one after old tokens expire
  • The verifier gets verification power only, never signing power, so compromising it cannot forge tokens
  • Adding callers does not scatter more secrets; the public key being public is the design intent
  • Beyond the signature you must check iss, aud and exp — skipping aud opens your API to tokens signed for someone else
  • Cache the key set but refetch on an unknown kid; HS256 is still reasonable when one service signs and verifies its own tokens
• 跨服务调用的幂等键该怎么设计？由谁生成、存在哪、重复了返回什么？How do you design an idempotency key for cross-service calls — who generates it, where does it live, and what do you return on a repeat?

  国内高频海外高频进阶#idempotency#distributed-systems#api-design

  分析过程 · 先想清楚再作答

  1. 这题的区分度全在实现细节上。概念谁都会说，能不能答对「谁生成、存在哪、返回什么」这三个具体问题，直接暴露你有没有真做过。
  2. 先立一条铁律：**幂等的最终裁判必须是数据库的唯一约束**，不是应用层的「先查一下有没有」。先查后插在单进程测试里永远是对的，一上多实例就出双份——两个副本同时查、同时发现没有、同时插入，这个时间窗压测时窄到复现不出来，上线后每天出几条脏数据。
  3. 再答「谁生成」：由**调用方**生成，因为只有它知道重试的那两次是同一件事；但键必须由事件内容决定，不能是每次重试重新生成的随机 UUID——那等于没有幂等。这条和 D8 的用户消息幂等是同一条判据。
  4. 跨服务比同服务多一个坑，这是本题最有价值的一点：**调用方给的 id 不能直接当键用**。两个不同的调用方各自造出 evt-1 是迟早的事，撞车之后的表现不是报错，而是后来那个用户静默收不到消息——他的事件被当成重复丢掉了，日志里干干净净。所以落库前要加命名空间，用「签发方 + 用户 id + 事件 id」三段拼，而且三段都取自验签后的令牌，伪造不了。
  5. 「返回什么」也是个坑：重复送达要返回 200 并附上第一次的结果，不要返回 409。重复不是错误，是分布式系统的常态；回 409 会让调用方的重试逻辑把它当失败处理，越重试越乱。
  6. 可以预期的追问：这张表会不会无限涨？答「会，所以要有保留期」——按业务能接受的重放窗口设一个 TTL（比如 7 天）定期清理，同时说明清理之后超期的重复请求会被当成新事件，这是一个明确的、可接受的取舍，不是漏洞。

  How to reason about it · think before answering

  1. This question separates people entirely on implementation detail. Anyone can define idempotency; answering who generates the key, where it lives, and what a repeat returns shows whether you have actually built one.
  2. Start with the rule: the final arbiter must be a database uniqueness constraint, not an application-level check-then-insert. Check-then-insert always passes single-process tests and produces duplicates the moment you run two replicas — both check, both find nothing, both insert. The window is too narrow to reproduce under load testing and wide enough to produce dirty rows daily in production.
  3. Who generates it: the caller, because only the caller knows that two retries are the same event. But the key must be derived from the event itself, never a fresh random UUID per retry — that is idempotency in name only. Same criterion as the user-message case from day 8.
  4. Cross-service adds one trap worth the most points: never use the caller's raw id as the key. Two different callers will eventually both produce evt-1, and the failure is not an error — the second user silently receives nothing, because their event is treated as a duplicate and the logs look clean. Namespace it: issuer plus user id plus event id, all three taken from the verified token so none of them can be forged.
  5. What to return also matters: a repeat gets 200 with the original result, not 409. Repeats are normal in distributed systems; a 409 makes the caller's retry logic treat it as a failure and the situation compounds.
  6. Expect: does this table grow forever? Yes, so give it a retention window — a TTL matching the replay window the business tolerates, say seven days, with periodic cleanup. Say plainly that a duplicate arriving after cleanup is treated as new; that is a stated trade-off, not a hole.

  答题要点

  • 最终裁判是数据库唯一约束加 on conflict do nothing，先查后插在多实例下必然出双份
  • 键由调用方生成，但必须由事件内容决定，随机 UUID 等于没有幂等
  • 调用方给的 id 不能直接当键：加命名空间（签发方 + 用户 id + 事件 id），三段都取自验签后的令牌
  • 撞车的后果不是报错而是另一个用户静默收不到消息，日志里看不出异常
  • 重复送达返回 200 加第一次的结果，不要返回 409，否则调用方会当失败继续重试
  • 幂等表要设保留期，超期后的重复会被当成新事件，这是明确取舍不是漏洞

  Key points

  • The arbiter is a unique constraint plus on conflict do nothing; check-then-insert duplicates as soon as you run two replicas
  • The caller generates the key, but it must be derived from the event — a fresh UUID per retry is not idempotency
  • Never use the caller's raw id: namespace it with issuer plus user id plus event id, all taken from the verified token
  • A collision does not raise an error; it silently drops another user's event and leaves clean logs
  • Return 200 with the original result on a repeat, never 409, or the caller's retry logic treats success as failure
  • Give the table a retention window and state that post-cleanup repeats count as new events — a stated trade-off, not a hole
• 设计一组给外部服务调用的 Agent 平台接口，你会怎么划分职责边界？You are designing the API surface an Agent platform exposes to other services. How do you draw the responsibility boundaries?

  国内高频海外高频深入#api-design#security#architecture

  分析过程 · 先想清楚再作答

  1. 这是开放题，考的是你有没有一条能反复用的划分依据。上来就罗列接口清单的人会被追问到没词；先给依据再给清单的人，追问反而是加分机会。
  2. 给一条判据：**按「谁拥有这份数据」划，不按「谁调用它」划。** 会话、执行记录、记忆、成本台账都属于平台，所以平台开的三个口子恰好是「写一条进来（inject）」「读写记忆（memory）」「查账（usage）」；编排逻辑属于对方，平台就不该提供「帮我跑一遍这个图」的接口——那是把对方的职责搬到自己身上，将来两边都改不动。
  3. 第二条判据是**贯穿全课的安全不变量：身份只能来自令牌，不能来自请求体**。所有接口都不接受 userId 参数，服务端一律从令牌的 sub 取。这条一旦破例，权限模型就整个塌了：查成本的接口如果接受 userId 查询参数，任何一张有效令牌都能遍历所有人的消费金额。同一条原则在 D12 的记忆检索工具上也出现过——不给模型身份参数，服务端自己填。
  4. 第三条是**返回粒度要按最小必要给**。usage 只返回汇总不返回明细，因为明细里带着执行 id 和模型选型，等于把平台的内部策略一并交出去；memory 要支持按 query 检索并限制条数，不提供「把这个人的所有记忆倒出来」的接口——一旦提供，它迟早会被某个图省事的调用方用成默认写法。
  5. 第四条是**每个写接口都要能被安全重放**：带 externalId、唯一约束兜底、重复返回 200。跨服务调用一定会重复，这不是要不要做的问题。
  6. 可以预期的追问：那限流按什么维度做？答「每用户，不是每调用方」——按调用方限流的话，一个用户的异常重试会把所有人的额度吃光；另外写接口要防回环，注入的消息要打来源标记，否则两个服务能把彼此拉进无限循环，账单是唯一会提醒你的东西。

  How to reason about it · think before answering

  1. This is an open design question testing whether you have a reusable criterion. Candidates who start listing endpoints run out of material under follow-ups; candidates who give the criterion first turn follow-ups into extra points.
  2. Offer the criterion: draw boundaries by who owns the data, not by who calls it. Sessions, run records, memories and the cost ledger belong to the platform, so the platform exposes exactly three things — write one event in (inject), read and write memory, and read usage. Orchestration belongs to the caller, so the platform should not offer run this graph for me; that pulls someone else's responsibility inside your walls and freezes both sides.
  3. Second criterion, the security invariant that runs through the whole course: identity comes from the token, never from the request body. No endpoint accepts a userId; the server always reads sub. Break this once and the authorization model collapses — a usage endpoint that accepts a userId query parameter lets any valid token enumerate everyone's spend. The same principle appeared on the memory search tool: the model gets no identity parameter, the server fills it in.
  4. Third, return the minimum necessary. Usage returns aggregates, not line items, because line items carry run ids and model choices — that hands over your internal strategy. Memory supports a query with a result limit rather than dump everything this user ever said; once that exists, some caller in a hurry will make it the default.
  5. Fourth, every write endpoint must be safely replayable: an externalId, a uniqueness constraint underneath, and 200 on a repeat. Cross-service calls will be duplicated; this is not optional.
  6. Expect: what dimension do you rate-limit on? Per user, not per caller — limiting per caller lets one user's runaway retries consume everyone's budget. Also guard against loops: tag injected messages with their source, or two services can pull each other into an infinite cycle and the bill is the only thing that tells you.

  答题要点

  • 按「谁拥有这份数据」划边界，不按「谁调用」划：会话、记忆、台账属于平台，编排属于对方
  • 三个口子对应三种所有权：inject 写入、memory 读写、usage 查账；不提供「帮我跑图」这种越界接口
  • 所有接口都不接受 userId 参数，身份一律从令牌 sub 取——这条破例一次权限模型就塌了
  • 返回粒度按最小必要：usage 只给汇总不给明细，memory 按 query 限条数而不是全量倒出
  • 每个写接口都带 externalId 并由唯一约束兜底，重复返回 200
  • 限流按每用户而不是每调用方；注入的消息要打来源标记防止两个服务互相回环

  Key points

  • Draw boundaries by data ownership, not by caller: sessions, memory and the ledger belong to the platform, orchestration belongs to the caller
  • Three endpoints for three kinds of ownership — inject, memory, usage — and no run this graph for me endpoint that crosses the line
  • No endpoint accepts a userId; identity always comes from the token's sub, and one exception collapses the model
  • Return the minimum necessary: usage gives aggregates only, memory takes a query with a limit instead of dumping everything
  • Every write endpoint carries an externalId backed by a uniqueness constraint and answers 200 on repeats
  • Rate-limit per user rather than per caller, and tag injected messages with their source so two services cannot loop forever

D20 定时 Job 与主动关怀：时区、quiet hours、每日上限、通知 provider 抽象

• 系统主动发给用户的消息，和用户自己触发的消息，在系统设计上有什么不同？How does a system-initiated message differ from a user-triggered one, from a system design point of view?

  国内高频海外高频基础#proactive-messaging#system-design#product-engineering

  分析过程 · 先想清楚再作答

  1. 这题看着像概念题，其实是筛人题。答「都是发消息，只是触发方不同」就落进了最浅的一层——面试官想听的是这个差别会逼你多写哪些代码。
  2. 先给三条结构化的差别：谁在等（用户触发时他正盯着屏幕，主动消息没有人在等）；失败怎么处理（用户触发的失败必须报错给他看，主动消息的失败多数时候应该安静地推迟或放弃）；凭什么发（用户触发是他开了口，主动消息你得自己说出理由）。
  3. 第三条是题眼，要说透：主动消息的默认答案是不发。每一条都要能回答为什么是现在、为什么是这个用户、为什么这条内容值得打断他，三个问题答不上任何一个就不该发。
  4. 然后给出这个差别在系统里的落点：主动消息这一侧必须多出一层准入判断，本课叫三道闸——按用户时区算本地时间、安静时段命中就推迟、每日上限满了就拦下。用户触发那一侧完全不需要这层。
  5. 代价也要算清楚，这是区分「读过文章」和「做过系统」的地方：用户对主动消息的容忍度极低，连着几条无关紧要的推送之后他不会争论内容对不对，直接关掉通知权限——而权限一关，你连真正重要的那条也送不出去了。你消耗的是一个用完就拿不回来的额度。
  6. 可以预期的追问：那定时任务和主动消息是不是一回事？答不是。定时任务解决的是「能按时触发」（中心调度、幂等键锚在计划触发的那一分钟），主动消息解决的是「该不该发」，前者是机制、后者是准入，两层要分开做。

  How to reason about it · think before answering

  1. This looks like a definition question but it is really a filter. Answering both send a message, only the trigger differs stays at the shallowest layer — the interviewer wants to know what extra code the difference forces you to write.
  2. Give three structured differences: who is waiting (a user-triggered reply has someone staring at the screen, a proactive message has nobody waiting); how failure is handled (user-triggered failures must surface as errors, proactive failures should usually be silently deferred or dropped); and what justifies sending (the user asked, versus you having to justify it yourself).
  3. The third is the hinge, so make it explicit: the default answer for a proactive message is do not send. Every one must answer why now, why this user, and why this content is worth interrupting them. Fail any of the three and it should not go out.
  4. Then land the difference in the system: the proactive path needs an admission layer the reactive path does not — compute the user's local time from their timezone, defer if it falls inside quiet hours, drop if the daily cap is used up.
  5. Quantify the cost, which is what separates having read about this from having shipped it: tolerance for proactive messages is very low. After a few irrelevant pushes the user will not argue about the content, they will revoke the notification permission — and once revoked, the genuinely important message cannot reach them either. You are spending a budget that never refills.
  6. Expect: so is a cron job the same thing as a proactive message? No. The scheduler solves firing on time (central scheduling, an idempotency key anchored to the scheduled minute); proactive care solves whether to send at all. One is mechanism, the other is admission, and they belong in separate layers.

  答题要点

  • 三条差别：谁在等、失败怎么处理、凭什么发；第三条是关键
  • 主动消息的默认答案是不发，每条要能回答为什么是现在、为什么是这个用户、为什么值得打断他
  • 落到系统上就是多一层准入判断：时区换算、安静时段、每日上限，用户触发那一侧不需要
  • 代价是一个不可再生的额度：推送惹烦了用户，他关掉权限之后重要消息也送不出去
  • 定时机制（能按时触发）和主动关怀（该不该发）是两层，不要混在一起做

  Key points

  • Three differences: who is waiting, how failure is handled, and what justifies sending — the third is the crux
  • The default answer for a proactive message is no; each one must justify why now, why this user, why worth interrupting
  • In the system this becomes an admission layer — timezone, quiet hours, daily cap — that the reactive path does not need
  • The cost is a non-renewable budget: annoy the user and they revoke notifications, taking the important messages down with them
  • Scheduling (fire on time) and proactive care (should we send) are two separate layers
• 做一个面向全球用户的定时推送服务，时区上你会踩到哪些坑？夏令时切换那天怎么处理？Building a scheduled push service for users worldwide, what timezone pitfalls would you hit, and how do you handle the DST switchover day?

  国内高频海外高频深入#timezone#scheduling#correctness

  分析过程 · 先想清楚再作答

  1. 这题的区分度全在夏令时那半句。只答「存 UTC、展示转本地」是及格线，能不能给出夏令时那天的**可验证裁定**决定了你是「知道有坑」还是「填过坑」。
  2. 先把基础两条说死：时间一律存 UTC（存的是绝对时刻），判断前先转成用户本地时区（判断的是墙上时间）。判据是一句可自查的话——只靠这一列加上用户档案里的时区，能不能唯一还原出那个绝对时刻。存本地时间字符串答不上来。
  3. 第二个坑是时区字段本身：必须存 IANA 标识（Asia/Shanghai）而不是 UTC 偏移量。偏移量一年会随夏令时变两次，标识是规则、偏移量只是规则在某一天算出来的结果，存结果的那一刻它就过期了。
  4. 第三个坑是夏令时那天的两种反常：春季前跳会让某个本地时间**根本不存在**（纽约 2026-03-08 的 02:30），秋季回拨会让某个本地时间**出现两次**（2026-11-01 的 01:30）。只要你的调度点落在这两个窗口里，「每天早上 8 点发」就解释不出唯一答案。裁定要显式给出而不是交给库随便选：不存在就顺延到过渡之后（02:30 变 03:30），出现两次就取第一次——这也正是 java.time 的 ZonedDateTime.of 的默认行为，可以直接写成断言测试。
  5. 第四个坑最容易被漏：用户跨时区旅行时，他的时区以哪一次为准。答案是以用户档案里那个显式字段为准、绝不跟着设备静默漂移；设备上报只用来询问是否切换。更细一层可以补：时区在 24 小时内跳变超过 3 小时时，当天的安静时段按新旧两个时区的并集处理，任何一边在安静就不发——保守的代价是晚几小时收到，激进的代价是在人家凌晨三点响一声。
  6. 可以预期的追问：这种 bug 为什么很难被测出来？因为本机、CI、生产常常都在同一个时区甚至都在 UTC，「忘了转时区」在所有测试里都是绿的。给出判据：把测试用户的时区故意设成三个互不相同、且都不等于服务器时区的值，任何依赖服务器时区的判断当场变红。

  How to reason about it · think before answering

  1. All the signal in this question lives in the DST half. Store UTC, render local is the passing grade; giving a verifiable ruling for the switchover day is what separates knowing the pitfall from having fixed it.
  2. Nail the two basics first: always store UTC (an absolute instant) and convert to the user's zone before any judgement (a wall-clock time). The self-check is one sentence — can this column plus the user's stored timezone uniquely reconstruct the absolute instant? A local time string cannot.
  3. The second pitfall is the timezone field itself: store the IANA identifier (Asia/Shanghai), never a UTC offset. Offsets shift twice a year under DST; the identifier is the rule and the offset is only what that rule evaluated to on one particular day, so it is stale the moment you persist it.
  4. The third is the two anomalies on switchover day: spring-forward makes some local time simply not exist (02:30 on 2026-03-08 in New York), and fall-back makes some local time occur twice (01:30 on 2026-11-01). If your schedule point lands in either window, send at 8am local has no unique answer. State the ruling explicitly rather than leaving it to whatever the library picks: shift a nonexistent time forward past the transition (02:30 becomes 03:30), and take the first occurrence when it happens twice — which is exactly what java.time's ZonedDateTime.of does, so you can assert on it in tests.
  5. The fourth is the one people miss: when a user travels across zones, which timezone counts. Answer: the explicit field on the user profile, never silent drift from device reports; a device report should only prompt the user to confirm a change. Go one level deeper if you can — when the zone jumps more than three hours within 24 hours, treat that day's quiet hours as the union of the old and new zones and stay silent if either is quiet. Being conservative costs a few hours of delay; being aggressive costs a 3am buzz.
  6. Expect: why are these bugs so hard to catch? Because your laptop, CI and production are often all in one zone, frequently UTC, so forgot to convert stays green everywhere. Give the fix: pin the test users to three distinct zones, none equal to the server's, and every server-timezone dependency turns red immediately.

  答题要点

  • 存储一律 UTC，判断前转用户本地时区；自查判据是这一列加时区能否唯一还原绝对时刻
  • 时区存 IANA 标识而不是 UTC 偏移量——偏移量随夏令时一年变两次，存下来就过期
  • 夏令时两种反常：本地时间不存在（春季前跳）、本地时间出现两次（秋季回拨）
  • 裁定要显式且可断言：不存在就顺延到过渡之后，出现两次取第一次（与 java.time 默认一致）
  • 跨时区旅行以用户档案里的显式字段为准，不跟设备漂；跳变超过 3 小时时按新旧时区的并集判安静
  • 测试里把用户时区设成三个不同于服务器的值，否则「忘了转时区」在所有测试里都是绿的

  Key points

  • Store UTC everywhere, convert to the user's zone before judging; the check is whether column plus zone reconstructs the instant
  • Persist IANA identifiers, not UTC offsets — offsets change twice a year and are stale on write
  • Two DST anomalies: a local time that does not exist (spring forward) and one that occurs twice (fall back)
  • Make the ruling explicit and assertable: shift nonexistent times past the transition, take the first of a duplicated pair (matching java.time)
  • For travellers, trust the explicit profile field, not device drift; on jumps over three hours, treat quiet hours as the union of both zones
  • Pin test users to three zones different from the server's, or the missing conversion stays green in every test
• quiet hours 和每日发送上限这两条规则你会怎么实现？它们应该放在链路的哪一步判断？How would you implement quiet hours and a per-user daily cap, and where in the pipeline should they be evaluated?

  国内高频海外高频进阶#rate-limiting#quiet-hours#cost-control

  分析过程 · 先想清楚再作答

  1. 这题有两个题眼，很多人只答了前一个。第一个是「怎么判断」（细节题），第二个是「放在哪一步」（架构题），后者才是拿分点。
  2. 先讲 quiet hours 的判断。把时刻折成从午夜起算的分钟数之后，绝大多数人第一次都会写成 start 小于等于 now 且 now 小于 end。这对午休那种同日区间是对的，对 22:00 到 08:00 恒为 false——start 是 1320、end 是 480，条件永远不成立，于是半夜照发。正确写法是 start 小于 end 时用「且」，start 大于 end（跨午夜）时换成「或」。这个 bug 恶劣在只在跨午夜的配置上错，用 13:00 到 14:00 写的单元测试全绿。
  3. 接着是命中之后怎么办：推迟，不是丢弃。判据不该由发送方临时决定，而应该由消息自己带一个过期时刻——过期时刻早于窗口结束的丢弃，其余一律推迟到窗口结束。限时取消提醒过了今晚就没意义，账单提醒明早发一样有效。另外要提一句惊群：所有推迟的消息会算出同一个到期时刻，要加一个按用户标识哈希得出的抖动（不能用随机数，否则线上复现不了）。
  4. 再讲每日上限的两个细节。一是「一天」必须是**用户本地日历日**，写成 UTC 日的话东八区用户早上八点前发的会算进昨天的额度。二是必须先占坑再判断——原子自增拿返回值比上限，超了再把名额还回去；先查后写在并发下两条候选会同时读到同一个值然后一起发出去。渠道明确拒绝时也要把名额还回去。
  5. 最后是架构题那一半，也是最值钱的一段：三道闸必须在**生成内容之前**判断，不是在发送那一步。顺序错了程序照样跑通、发出的消息也一样，唯一区别是每条被拦下的消息你都已经付过一次模型调用的钱。按 1000 用户每天各 3 条候选、拦掉四成、单条约 0.00075 美元算，一个月白花约 27 美元，比这批用户的正常对话开销还高，而且监控上完全看不出来——只有把候选数和实际发送数并排摆出来才看得见差额。
  6. 可以预期的追问：三道闸内部谁先谁后？答时区、安静时段、每日上限，上限必须最后。因为被安静时段推迟的消息明早才发，不该占掉今天的名额；顺序反了用户会发现自己明明没收到几条却被限流了。

  How to reason about it · think before answering

  1. There are two hinges here and most candidates only answer the first. One is how to evaluate the rules (a details question), the other is where in the pipeline (an architecture question) — the second is where the points are.
  2. Start with quiet hours. Once you fold times into minutes-from-midnight, almost everyone first writes start less-or-equal now and now less-than end. That is correct for a same-day window like a lunch break, but for 22:00 to 08:00 it is always false: start is 1320, end is 480, the condition never holds, and you push at 3am. The fix is to use and when start is before end, and or when start is after end. What makes this bug nasty is that it only misfires on the cross-midnight config, so a unit test written around 13:00 to 14:00 passes.
  3. Then what to do on a hit: defer, do not drop. The decision should not be the sender's mood — attach an expiry to each candidate and drop only when it expires before the window ends, deferring everything else to the window's end. A thirty-minute cancellation warning is worthless tomorrow; a billing summary is just as valid at 8am. Mention the thundering herd too: every deferred message resolves to the same due instant, so add jitter derived from a hash of the user id, never a random number, or you cannot reproduce incidents.
  4. Now two details on the daily cap. First, the day must be the user's local calendar day; keying on the UTC date charges an East-Asian user's 8am message to yesterday's budget. Second, increment first and check the returned value, then give the slot back if it exceeded — a read-then-write races, letting two candidates read the same count and both go out. Return the slot on a hard rejection from the channel as well.
  5. Finish with the architecture half, which is the valuable part: all three gates must run before the content is generated, not at the send step. Get the order wrong and the program still works and sends the same messages; the only difference is that you paid for a model call on every message you then threw away. At 1000 users, three candidates each per day, forty percent blocked and roughly $0.00075 per message, that is about $27 a month wasted — more than the normal conversational spend for the same cohort — and it is invisible in monitoring. Only putting candidate count next to sent count reveals the gap.
  6. Expect: what order do the three gates run in? Timezone, quiet hours, daily cap, with the cap last. A message deferred to tomorrow morning must not consume today's quota; reverse the order and users get rate-limited despite having received almost nothing.

  答题要点

  • 跨午夜的安静时段：start 小于 end 用「且」，start 大于 end 换成「或」，朴素写法对 22:00-08:00 恒为 false
  • 命中安静时段是推迟到窗口结束而不是丢弃；只有自带的过期时刻早于窗口结束才丢
  • 推迟会造成惊群，要加按用户标识哈希得出的抖动，不能用随机数
  • 每日上限的「天」必须是用户本地日历日，不是 UTC 日
  • 计数要先占坑再判断（原子自增后比上限，超了还回去），先查后写在并发下会超发
  • 三道闸必须在生成内容之前判断，装晚了每条被拦的消息都已经付过模型调用的钱（示例量级约 27 美元每月）；闸内顺序是时区、安静时段、每日上限，上限最后

  Key points

  • Cross-midnight quiet hours need or when start is after end; the naive and version is always false for 22:00-08:00
  • On a hit, defer to the end of the window rather than drop; only drop when the message's own expiry precedes that
  • Deferral causes a thundering herd, so add jitter hashed from the user id, never a random value
  • The day in a daily cap must be the user's local calendar day, not the UTC date
  • Increment atomically then compare and release on overflow; read-then-write over-sends under concurrency
  • Run all three gates before generating content — otherwise every blocked message has already been paid for (about $27/month at the example scale); order them timezone, quiet hours, daily cap, with the cap last
• 模型调用、支付、通知渠道你都做过 provider 抽象。通知这一份接口和另外两份有什么不同？You have abstracted model calls, payments and notification channels behind providers. How does the notification interface differ from the other two?

  国内高频海外高频进阶#provider-abstraction#api-design#retry-semantics

  分析过程 · 先想清楚再作答

  1. 这题在考你是「会套模式」还是「懂模式」。把三者说成一回事——都是接口加多个实现、换厂商不改业务——只答到了共性那一层，面试官真正想看的是你有没有识别出差异并把它写进接口。
  2. 先给共性，一句话带过：都是把「会被替换的东西」推到接口后面，业务代码只认接口，选择点集中在一处。这一层是对的，但不构成区分度。
  3. 然后给三条差异，这是拿分点。第一，收下不等于送达：支付网关返回成功钱就划走了，通知渠道返回成功只表示它收下了，真正送达是过一会儿通过回执异步告诉你的。所以返回值只能叫 accepted 不能叫 delivered，而且必须带渠道侧的消息标识，回执回来时靠它对上号。
  4. 第二，限流的层次不同：渠道自带每秒条数上限并会用 429 加重试间隔告诉你稍后再来，这是**渠道维度的技术约束**；而每日发送上限是**用户维度的礼貌约束**。两者混成一个概念就没法分别调整——一个说的是这条线路挤不下了，一个说的是这个人今天已经被打扰够了。
  5. 第三，没有撤销：支付有退款，通知发出去就撤不回来，取消只在交给渠道之前有效。所以接口里不能出现 cancel——在接口上留一个做不到的操作比根本没有这个操作更危险，调用方会真的去用它。
  6. 可以预期的追问：那失败重试怎么设计？答分三类：限流按渠道给的时长退避重试；参数错（正文非法、用户已退订）不可重试，直接放弃并把当天的名额还回去；服务端错误或超时可重试但必须带同一个幂等键——数据库里多一行你能删掉，用户手机上多响一声删不掉。再补一条判断抽象好坏的判据：新接一个渠道时如果它只需要实现「把这条消息发出去」，抽象就对了；如果它还得知道现在是不是安静时段、这是今天第几条，说明业务规则泄进了渠道层。

  How to reason about it · think before answering

  1. This question separates applying a pattern from understanding one. Saying all three are the same — an interface with several implementations so you can swap vendors without touching business code — only covers the shared part; the interviewer wants to see whether you spotted the differences and encoded them in the interface.
  2. Acknowledge the commonality in one line: each pushes a replaceable dependency behind an interface, business code depends only on the interface, and the selection point lives in exactly one place. Correct, but not differentiating.
  3. Then give three differences, which is where the points are. First, accepted is not delivered: when a payment gateway returns success the money has moved, but when a notification channel returns success it has merely taken the message, and actual delivery arrives later as an asynchronous receipt. So the result is accepted, never delivered, and it must carry the provider-side message id so the receipt can be correlated.
  4. Second, throttling lives at a different layer: the channel has its own per-second ceiling and tells you to come back later with a 429 plus a retry interval — a channel-level technical constraint — while the daily cap is a user-level courtesy constraint. Collapsing them into one concept makes them impossible to tune separately: one says this line is congested, the other says this person has been interrupted enough today.
  5. Third, there is no undo: payments have refunds, notifications do not. Once handed to the channel the message is gone, and cancel only means anything before that handoff. So the interface must not expose a cancel method — leaving an operation that cannot work is worse than not having it, because callers will actually use it.
  6. Expect: how do you design retries then? Three classes. Throttling backs off for the interval the channel gave you. Parameter errors (invalid body, unsubscribed user) are not retryable, so give up and return the daily slot. Server errors and timeouts are retryable but must carry the same idempotency key — you can delete a duplicate row, you cannot un-buzz a phone. Add a test for the abstraction itself: if a new channel only has to implement send the message, the boundary is right; if it also needs to know whether it is quiet hours or which message of the day this is, business rules have leaked into the channel layer.

  答题要点

  • 共性是把可替换依赖推到接口后面、选择点集中一处，但这只是及格线
  • 收下不等于送达：返回值叫 accepted 不叫 delivered，必须带渠道侧消息标识以便异步回执对号
  • 限流分两层：渠道的每秒上限是技术约束，每日发送上限是用户维度的礼貌约束，不能合并
  • 通知没有撤销，接口里不能有 cancel；留一个做不到的操作比没有更危险
  • 重试分三类：限流按渠道给的时长退避、参数错不可重试并归还名额、服务端错误可重试但必须带同一个幂等键
  • 判断抽象切没切对：新渠道只需实现发送就对了，还要知道安静时段和当天条数就说明业务泄进了渠道层

  Key points

  • The shared part is pushing a replaceable dependency behind an interface with a single selection point — that is only the baseline
  • Accepted is not delivered: name the result accepted and carry a provider message id so async receipts can be correlated
  • Throttling has two layers: the channel's per-second ceiling is technical, the daily cap is a user-level courtesy rule, and they must stay separate
  • Notifications have no undo, so the interface must not expose cancel — an unimplementable operation is worse than none
  • Three retry classes: back off for the channel's interval on throttling, give up and release the slot on parameter errors, retry server errors with the same idempotency key
  • Test the boundary: a new channel should only implement send; needing to know quiet hours or today's count means business rules leaked into the channel

D21 评估与可观测：golden set、LLM-as-judge、tracing、失败率/成本面板；Pi vs LangGraph 总结；W3 复盘

• 怎么评估一个 Agent 的效果？和传统后端服务的测试有什么不同？How do you evaluate an agent's quality, and how does it differ from testing a conventional backend service?

  国内高频海外高频进阶#evaluation#testing#agent-quality

  分析过程 · 先想清楚再作答

  1. 题眼是「不同」。只答「建一个测试集跑准确率」拿不到分——那是机器学习的标准答案，面试官想看你能不能说清 Agent 这个场景特殊在哪。
  2. 根子上的差别只有一句：**同样的输入，Agent 不保证给同样的输出**。传统测试的断言是「等于」，而 Agent 的产出没有唯一正确答案，只有「够不够好」。断言从等值变成了判分，整套方法论跟着变。
  3. 这条差别连锁出三个后果，说全了这题就稳了：一是**跑没跑通判断不了质量**——流程没抛错，不等于回复里写明了退款结论；二是**改动的影响是弥散的**，改一个字的提示词可能只影响一类请求，人肉抽查五条恰好没覆盖到，你会得出「没影响」然后上线；三是**多 Agent 又难一层**，一次请求走路由、拆分、并行执行、评审、汇总五道工序，任何一道歪了都表现成「最后那段话不太对」，不分开量就不知道该改哪块。
  4. 所以给出定位：**评估不是测试，评估是给一个随机系统建立一条可比较的基线。** 它的产物不是「通过」或「不通过」，而是一个能和上一次比的数字。既然要比，样本集就必须固定。
  5. 然后落到具体做法：一个小而稳的 golden set（本课 15 条），每条写清期望走哪条路由和一份必备信息清单；用 LLM-as-judge 对照清单打分；把评估结果和链路追踪接到同一块面板上。**清单是关键**——它把「这答得好吗」这种没法验的问题，换成了「这几件事写没写」这种能验的问题。
  6. 可以预期的追问：那还需要单元测试吗？需要，而且分工很清楚——工具函数、状态迁移、reducer 这些确定性的部分照旧用单元测试断言等值，评估只负责模型产出那一段。把两者混成一套，你会得到一堆随机失败的测试，然后所有人开始无视 CI。

  How to reason about it · think before answering

  1. The hinge is differ. Answering build a test set and measure accuracy is the textbook ML answer and misses the point; the interviewer wants to know whether you can articulate what makes agents special here.
  2. The root difference is one sentence: the same input does not guarantee the same output. Conventional tests assert equality, but an agent's output has no single correct answer, only good enough. Once the assertion changes from equality to scoring, the whole methodology changes with it.
  3. That difference cascades into three consequences, and covering all three secures the question. First, whether it ran tells you nothing about quality — the flow not throwing does not mean the reply stated the refund conclusion. Second, the blast radius of a change is diffuse: a one-word prompt edit may affect only one class of request, and hand-checking five samples that happen to miss that class yields no impact, then you ship. Third, multi-agent adds a layer: one request passes routing, planning, parallel execution, review and aggregation, and any one of them going wrong surfaces as that last paragraph seems off — without measuring each stage you cannot tell which to fix.
  4. So frame it: evaluation is not testing. Evaluation establishes a comparable baseline for a stochastic system. Its output is not pass or fail but a number you can compare against last time — and to compare, the sample set must be frozen.
  5. Then get concrete: a small stable golden set (15 items here), each declaring its expected route and a checklist of facts the reply must contain; an LLM-as-judge scoring against that checklist; and evaluation results joined to tracing on one dashboard. The checklist is the key move — it converts is this a good answer, which cannot be verified, into were these facts stated, which can.
  6. Expect: do you still need unit tests? Yes, with a clean split — deterministic parts (tool functions, state transitions, reducers) keep asserting equality in unit tests, while evaluation covers only the model-generated segment. Merge the two and you get a suite that fails randomly, after which everyone starts ignoring CI.

  答题要点

  • 根本差别：同样的输入 Agent 不保证同样的输出，断言从「等于」变成「够不够好」
  • 三个后果：跑通不等于质量合格、改动影响弥散（抽查会漏）、多 Agent 里五道工序任一歪了都表现成同一个症状
  • 定位：评估不是测试，是给随机系统建一条可比较的基线，产物是能和上次比的数字而不是通过与否
  • 做法：小而稳的 golden set + 每条的必备信息清单 + LLM-as-judge 打分 + 与 tracing 同源的面板
  • 清单是关键，它把「答得好吗」换成「这几件事写没写」，从没法验变成能验
  • 单元测试仍然需要，负责确定性部分；两者混在一起会让 CI 随机变红，最后被所有人无视

  Key points

  • The root difference: identical input does not guarantee identical output, so the assertion shifts from equality to good enough
  • Three consequences: running is not quality, change impact is diffuse (sampling misses it), and in multi-agent any of five stages failing looks like the same symptom
  • Framing: evaluation is not testing — it establishes a comparable baseline for a stochastic system, yielding a number rather than pass/fail
  • Method: a small stable golden set, a required-facts checklist per item, an LLM-as-judge, and a dashboard sharing tracing's data source
  • The checklist is the key move: it converts is this good into were these facts stated — unverifiable into verifiable
  • Unit tests remain for deterministic parts; merging the two makes CI fail randomly until everyone ignores it
• 用大模型给大模型的输出打分（LLM-as-judge），有哪些不可靠的地方？怎么办？What makes LLM-as-judge unreliable, and what do you do about it?

  国内高频海外高频深入#evaluation#llm-as-judge#reliability

  分析过程 · 先想清楚再作答

  1. 这题筛的是「你是真用过，还是听说过」。用过的人能报出具体的失效形态和量级，没用过的人只会说「可能不准」。
  2. 第一种，**同源偏差**：judge 和被评估的 Agent 用同一个模型时，它偏向认可自己的输出——同一个模型对「什么算好答案」的偏好是一致的，让它复核自己刚写的东西，它当然觉得没问题。实测数量级：同一批被改坏的产出，同源 judge 给 14/15，换个模型只给 12/15，被多放过去的正是最该抓的边缘产出。这个坑不止在 judge，**凡是「模型评模型」的位置都有**，Critic 节点是同一个问题。
  3. 第二种，**长度偏好**：judge 倾向给篇幅大的答案更高分。实测：一条 33 字的正确回复灌上 141 字无关客套话，凭印象打分的提示词就从 2 分涨到 4 分，内容一个字没变。
  4. 第三种，**评分提示词漂移**：judge 的评分随提示词微调整体移动。同一份产出，两套评分提示词一套给 2 分一套给 5 分。所以有条硬纪律——**分数只在同一套 judge 提示词内部可比**，跨版本比较是没有意义的。
  5. 解药要一一对应，别笼统说「多测几次」：固定 judge 提示词并版本化（每条评分记录带上 rubric 版本与 judge 模型，那是它的坐标；面板发现混了两套口径应当直接拒绝聚合，而不是算出一个没含义的平均分）；默认用不同的模型当 judge，而且这该是默认值不是可选项；留一小批人工标注做校准集，每次改评分提示词拿它对一遍，**比的是结论（过或不过）而不是分数差**——差 1 分无所谓，结论翻了就是事故。
  6. 还有一条更根本的：**把评分标准从主观印象换成可核对的清单**，它同时解掉长度偏好——照清单逐条数，灌水加不了分。实测那条灌水回复在清单口径下前后都是 5 分，纹丝不动。
  7. 可以预期的追问：judge 便宜还是人工便宜？答：judge 的成本和被评估的系统本身一个量级，所以「跑一次全量评估多少钱」是你决定每次提交都跑还是每天跑一次的依据；而人工的成本不在钱在延迟——它给不了你改一次提示词就想看一次结果的反馈速度，所以人工只该用在校准集上。

  How to reason about it · think before answering

  1. This screens for whether you have actually used it. People who have can name specific failure shapes with magnitudes; people who have not just say it might be inaccurate.
  2. First, self-preference: when the judge and the evaluated agent share a model, it favours its own output — the same model has a consistent notion of what a good answer looks like, so asking it to review what it just wrote gets an approving verdict. Measured: on the same batch of deliberately degraded outputs, a same-model judge gave 14/15 while a different model gave 12/15, and the extra passes were exactly the borderline cases worth catching. This is not confined to judges — every model-grading-model position has it, and a Critic node is the same problem.
  3. Second, length bias: judges reward longer answers. Measured: padding a correct 33-character reply with 141 characters of irrelevant pleasantries moved an impression-based rubric from 2 to 4 without changing a word of substance.
  4. Third, rubric drift: scores shift wholesale when the judge prompt is tweaked. The same output scored 2 under one rubric and 5 under another. Hence the hard rule: scores are comparable only within one judge prompt, and cross-version comparison is meaningless.
  5. Match each remedy to its failure rather than saying run it a few more times. Freeze and version the judge prompt — every score record carries its rubric version and judge model, which are its coordinates, and a dashboard that finds two rubrics mixed should refuse to aggregate rather than emit a meaningless average. Default to a different model as judge, as a default and not an option. Keep a small human-labelled calibration set and re-run it whenever the rubric changes, comparing verdicts (pass or fail) rather than score deltas — one point of drift is fine, a flipped verdict is an incident.
  6. And one deeper fix: replace impressionistic criteria with a checkable list, which also dissolves length bias — counting items off a list gives padding nothing to earn. Measured, that padded reply scored 5 both before and after under the checklist rubric.
  7. Expect: is a judge cheaper than humans? The judge's cost is the same order as the system being evaluated, so what a full evaluation run costs decides whether you run it per commit or nightly. Human cost is not money but latency — it cannot give you feedback at the speed of one prompt edit, which is why humans belong on the calibration set only.

  答题要点

  • 同源偏差：judge 与被评估 Agent 同模型会虚高（实测 14/15 vs 异源 12/15），且凡「模型评模型」的位置都有，Critic 同理
  • 长度偏好：灌水 141 字能让印象分从 2 涨到 4，内容一字未变
  • 评分提示词漂移：同一产出两套 rubric 一个 2 分一个 5 分，所以分数只在同一套提示词内部可比
  • 解药一一对应：rubric 版本化并随记录存坐标、面板发现混口径直接拒绝聚合、默认换模型当 judge
  • 留人工标注校准集，比结论（过/不过）而不是比分数差——差 1 分无所谓，结论翻了是事故
  • 更根本的是把主观印象换成可核对的清单，同时解掉长度偏好（清单口径下灌水前后都是 5 分）

  Key points

  • Self-preference: a same-model judge inflates scores (14/15 vs 12/15 cross-model), and it applies to every model-grading-model spot including Critic
  • Length bias: 141 characters of padding moved an impression score from 2 to 4 with no substantive change
  • Rubric drift: the same output scored 2 and 5 under two rubrics, so scores compare only within one judge prompt
  • Remedies map one-to-one: version the rubric and store it alongside each record, refuse to aggregate mixed rubrics, default to a different judge model
  • Keep a human-labelled calibration set and compare verdicts, not score deltas — a point of drift is fine, a flipped verdict is an incident
  • The deeper fix is a checkable list instead of impressions, which also removes length bias (the padded reply scored 5 both ways)
• 多 Agent 系统的可观测性要看哪些东西？和单 Agent 有什么不一样？What does observability look like for a multi-agent system, and how does it differ from a single agent?

  国内高频海外高频进阶#observability#tracing#distributed-systems

  分析过程 · 先想清楚再作答

  1. 题眼在「不一样」。答「加日志加监控」等于没答，要说清结构上的差别。
  2. 结构差别一句话：**单 Agent 的一次调用是一条线，多 Agent 是一棵树。** 一次请求走监督者路由、规划者拆三件、三个执行者并行、评审者打回一件、那件重跑、最后汇总——按时间平铺看不出谁在谁里面，也看不出哪两个是并行的。
  3. 所以 span 必须带**父指针**，这是全部关键：有它才是树，没它只是一张平铺列表，你知道发生过什么，却不知道谁触发了谁。一条 span 的字段少得出奇——id、父指针、名字、起止时刻、几个属性，就够还原整棵树。
  4. 父子关系怎么传下去也是个考点：**不要在每个函数上加一个 parentSpanId 参数**，每加一个节点都要改签名、漏一处断一截。用语言自带的隐式上下文——JS 的 AsyncLocalStorage、Python 的 contextvars、Swift 的 TaskLocal，Java 用 ScopedValue 或 ThreadLocal 配合线程池的显式传播。
  5. 然后说面板要回答哪四个问题：错了多少（通过率、路由准确率、降级率、兜底率）、慢在哪（p50/p95）、花了多少、**钱花在哪个角色身上**（按节点分摊）。最后一样是多 Agent 特有的，也最有用——实测执行者节点占了成本三分之一强，一眼就知道压成本先压哪儿。
  6. 还有一条地基性的：**面板不是另一套埋点，是 trace 的聚合**。同一份原始数据横着看是树、竖着堆是面板。两套数据来源迟早会对不上，然后没有人相信任何一个。
  7. 最后回指路由：路由决策是模型做的，同一句话下次未必给同样的答案，所以必须把**路由理由**一起记下来——当时不记，那次判断就永远丢了。这是多 Agent 里最容易漏、又最需要事后审计的一条。

  How to reason about it · think before answering

  1. The hinge is differ. Saying add logs and metrics is a non-answer; name the structural difference.
  2. In one sentence: a single agent's call is a line, a multi-agent request is a tree. One request goes supervisor routing, planner splitting into three, three executors in parallel, a critic rejecting one, that one rerunning, then aggregation — flattened by time you cannot see nesting or which two ran concurrently.
  3. So spans must carry a parent pointer; that is the whole game. With it you have a tree, without it a flat list where you know what happened but not what triggered what. A span needs surprisingly few fields — id, parent, name, start and end, a few attributes — to reconstruct the entire tree.
  4. How the parent propagates is itself an interview point: do not thread a parentSpanId parameter through every function, because each new node then changes a signature and one omission breaks the chain. Use the language's implicit context — AsyncLocalStorage in JS, contextvars in Python, TaskLocal in Swift, and ScopedValue or ThreadLocal with explicit propagation across thread pools in Java.
  5. Then the four questions a dashboard must answer: how much is wrong (pass rate, routing accuracy, degradation rate, fallback rate), where is it slow (p50/p95), what did it cost, and which role spent the money (cost attributed per node). That last one is multi-agent specific and the most actionable — measured, executor nodes took over a third of spend, telling you immediately where to optimise.
  6. One foundational point: the dashboard is not a second instrumentation layer, it is an aggregation of traces. The same raw data read across is a tree and stacked up is a dashboard. Two separate sources will eventually disagree, after which nobody trusts either.
  7. Finally, tie back to routing: the routing decision is made by a model and the same sentence may route differently next time, so the routing rationale must be recorded — if you do not capture it then, that judgement is gone forever. It is the easiest thing to omit and the thing most needing post-hoc audit.

  答题要点

  • 结构差别：单 Agent 一次调用是一条线，多 Agent 是一棵树（路由→拆分→并行执行→评审打回→重跑→汇总）
  • span 必须带父指针，否则只是平铺列表，看不出嵌套关系也看不出并行
  • 父子关系用语言自带的隐式上下文传（AsyncLocalStorage / contextvars / TaskLocal），不要在每个函数签名上加参数
  • 面板回答四个问题：错了多少、慢在哪、花了多少、钱花在哪个角色身上（最后一个是多 Agent 特有且最有用）
  • 面板必须是 trace 的聚合而不是另一套埋点，两套数据源迟早对不上
  • 路由理由必须记下来：路由是模型做的决策，当时不记那次判断就永远丢了

  Key points

  • Structural difference: a single agent call is a line, multi-agent is a tree (route, split, parallel execute, critic reject, rerun, aggregate)
  • Spans need a parent pointer, or you have a flat list showing neither nesting nor parallelism
  • Propagate parentage through implicit context (AsyncLocalStorage / contextvars / TaskLocal), not a parameter on every signature
  • The dashboard answers four questions: how much is wrong, where it is slow, what it cost, and which role spent it — the last is multi-agent specific and most actionable
  • The dashboard must be an aggregation of traces, not separate instrumentation; two sources will disagree
  • Record the routing rationale: routing is a model decision, and uncaptured it is lost forever
• 什么时候该用 LangGraph 这类编排框架，什么时候不该用？When should you reach for an orchestration framework like LangGraph, and when should you not?

  国内高频海外高频进阶#architecture#framework-selection#langgraph

  分析过程 · 先想清楚再作答

  1. 这题最怕答成特性对比表。面试官想听的是判据，而且是能反过来说「不该用」的判据——只会说该用的人，通常是没被框架坑过的人。
  2. 先给三条该拆的判据（一条都不命中就别拆，也别引框架）：**提示词里出现了互斥的行为要求**（既要严谨又要俏皮，调好一个另一个就坏）；**工具多到选错率明显上升**；**某一步需要独立的失败与重试语义**（比如查库存失败该重试，拟退款方案失败该转人工，两者不能共用一套策略）。
  3. 然后给框架本身的判据，核心是一句：**要让多个角色并行写同一份状态，就必须显式；不需要并行，显式就是纯负担。** LangGraph 的价值是把合并规则声明在字段上——三个执行者并行写同一个工作区，谁的写入怎么合并，这件事必须有地方声明。反过来，一两个工具、循环最多两轮的场景，一个 while 加一个 switch 就够了，引入框架是净亏。
  4. 对比 Pi 这类高层 SDK 时，用维度而不是特性：上手成本（Pi 默认值多所以快，代价是模型和人设都是它替你挑的）、状态管理的显式程度（Pi 的历史在会话内部你感知不到，所以想改「同一个工作区怎么合并」时根本没有位置可改）、调试形态（Pi 给事件流是一条时间线，LangGraph 给逐节点增量和检查点是一棵可回放可分叉的树——**线性问题看时间线更快，多角色问题必须看树**）。
  5. 跨语言这条值得单独提，因为它常被忽略：**Java 和 Swift 都没有 LangGraph**，跨语言团队要么统一到 TS/Python，要么自己手写同一套结构。选框架的时候把这条算进去，比上线后再发现便宜。
  6. 可以预期的追问：那你怎么选？给一句可执行的：**你更怕看不见的默认值，还是更怕写不完的样板？** 怕前者选显式框架，怕后者选高层 SDK。这句话比任何特性表都实用。

  How to reason about it · think before answering

  1. The trap is answering with a feature matrix. The interviewer wants criteria, specifically criteria that can also say do not use it — people who can only argue for adoption usually have not been burned by a framework.
  2. Start with three criteria for splitting at all (if none holds, do not split and do not add a framework): the prompt contains mutually exclusive behavioural demands (rigorous and playful at once, where tuning one breaks the other); tools have grown numerous enough that selection error is visibly rising; or some step needs its own failure and retry semantics (an inventory lookup should retry, a refund draft should escalate to a human, and they cannot share one policy).
  3. Then the framework criterion, which is one sentence: if multiple roles write the same state concurrently, it must be explicit; if you do not need concurrency, explicitness is pure overhead. LangGraph's value is declaring merge rules on the field — with three executors writing one workspace, how those writes combine has to be declared somewhere. Conversely, with two tools and a loop that runs at most twice, a while and a switch suffice and a framework is a net loss.
  4. When comparing against a higher-level SDK like Pi, use dimensions rather than features: onboarding cost (Pi's defaults make it fast, at the price of it choosing your model and persona); explicitness of state (Pi keeps history inside the session, so when you want to change how one workspace merges there is no place to change it); and debugging shape (Pi gives an event stream, one timeline; LangGraph gives per-node deltas and checkpoints, a replayable and forkable tree — linear problems read faster as a timeline, multi-role problems require the tree).
  5. Cross-language deserves its own mention because it is routinely forgotten: neither Java nor Swift has LangGraph, so a polyglot team either standardises on TS/Python or hand-writes the same structure. Pricing that in during selection is cheaper than discovering it after launch.
  6. Expect: so how do you choose? Give something actionable: do you fear invisible defaults more, or endless boilerplate more? Fear the former and pick the explicit framework; fear the latter and pick the high-level SDK. That sentence is more useful than any feature table.

  答题要点

  • 先答该不该拆：提示词有互斥的行为要求、工具多到选错率上升、某步需要独立的失败与重试语义——一条不命中就别拆也别引框架
  • 框架判据一句话：多个角色并行写同一份状态就必须显式；不需要并行，显式就是纯负担
  • LangGraph 的价值是把合并规则声明在字段上；Pi 的历史在会话内部，想改合并方式根本没有位置可改
  • 调试形态不同：事件流是一条时间线，逐节点增量加检查点是一棵可回放可分叉的树；线性问题看时间线，多角色问题必须看树
  • Java 和 Swift 都没有 LangGraph，跨语言团队要么统一栈要么手写同一套结构，选型时就要算进去
  • 一句可执行的选型判据：更怕看不见的默认值就选显式框架，更怕写不完的样板就选高层 SDK

  Key points

  • First decide whether to split at all: mutually exclusive prompt demands, rising tool-selection error, or a step needing its own retry semantics — none holding means no split and no framework
  • The framework criterion in one line: concurrent writes to shared state require explicitness; without concurrency, explicitness is pure overhead
  • LangGraph's value is declaring merge rules on the field; Pi keeps history inside the session, leaving nowhere to change merge behaviour
  • Different debugging shapes: an event stream is a timeline, per-node deltas plus checkpoints are a replayable forkable tree — timelines for linear problems, trees for multi-role ones
  • Neither Java nor Swift has LangGraph, so polyglot teams standardise or hand-write the structure — price that in at selection time
  • An actionable heuristic: fear invisible defaults, choose the explicit framework; fear endless boilerplate, choose the high-level SDK
• 系统设计：一个多 Agent 客服平台已经上线，团队每周改几次提示词，但没人说得清质量是变好还是变差，成本也只有一个月底的总数。请为它设计一套评估与可观测体系。System design: a multi-agent support platform is live, the team edits prompts several times a week, nobody can say whether quality is improving, and cost is only known as a month-end total. Design its evaluation and observability system.

  国内高频海外高频深入#system-design#evaluation#observability#cost

  分析过程 · 先想清楚再作答

  1. 先别画架构图。这道题的陷阱是它听起来像「搭一套监控」，于是很多人上来就报 Prometheus 加 Grafana——那答的是基础设施，不是这道题。花三到五分钟问清四件事：一是**改提示词的频率和发布方式**（每周几次、有没有灰度、能不能回滚）；二是**现在出问题是怎么发现的**（用户投诉？还是有人偶然看到？）；三是**有没有历史数据**（线上对话存了多久、能不能回放）；四是**谁来看这套东西**（工程师排障，还是老板看成本）。这四个答案会实质改变设计，问它们本身就是分数。
  2. 然后给主干，一句话定形状：**一份数据、两种读法。** 埋点只做一套（span），横着读是一次请求的调用树（排障用），竖着堆是面板（趋势和成本用）。**这条是地基**——两套数据来源迟早对不上，然后没有人相信任何一个。很多候选人在这里就分叉成「监控系统」和「评估系统」两套，那是后面所有麻烦的源头。
  3. 接着按三层展开。**第一层，离线回归**：建一个小而稳的 golden set（15 到 50 条），三层覆盖——每条路由都有人走、每种失败模式各一条（置信度不足落兜底、工具预算耗尽降级、下游挂掉）、以及历史上真出过事故的那几条。每条写清期望路由和必备信息清单。维护规矩是**只增不改**：改一条期望，历史分数全部作废。用 LLM-as-judge 对照清单打分，**judge 换一个模型、rubric 版本化并随每条记录存下来**。这一层挂在 CI 上，每次改提示词跑一遍，产出一个能和上次比的数字。
  4. **第二层，在线观测**：每次请求落一棵 span 树，必须记路由理由（模型做的决策，当时不记就永远丢了）、每个节点的 token 与耗时、以及降级和兜底事件。面板回答四个问题：错了多少、慢在哪、花了多少、**钱花在哪个角色身上**。最后一个是多 Agent 特有的，也最有用。
  5. **第三层，在线采样评估**：离线的 15 条覆盖不了真实流量分布，所以按比例采样线上请求（比如 1%）跑同一套 judge，得到一条真实质量曲线。**这一层是前两层的桥**：离线告诉你有没有改坏已知的东西，在线告诉你真实用户遇到了什么。
  6. 成本这块要给数字感，这是区分层级的地方。**多 Agent 一次用户请求可能产生 5 到 10 次模型调用**，所以「每次调用多少钱」比真实单价小一个数量级，**必须按请求算钱**。给个算式：日活一万、人均三次会话、每次 5 次调用就是 15 万次调用；按输入 2000 输出 500 token、$0.15/$0.60 每百万算，一天约 90 美元。这个数立刻推出两件事：按节点分摊能定位省钱的地方，以及**评估本身的成本要单独记**——judge 调用和被评估系统一个量级，它决定你每次提交都跑还是每天跑一次。
  7. 最后收在「怎么让它真的被用起来」，这是很多人漏的一层：把评估结果接进发布流程（通过率跌破阈值就挡住发布）、把 rubric 和 golden set 放进代码仓库走 code review、以及**给每个机制配一句失效模式**——judge 会偏向同源模型、golden set 会被针对性优化（有人为了让它绿而调提示词，那一刻它就失去了意义）、采样会漏掉长尾。说不出失效模式的方案，面试官会认为你只是读过。
  8. 可以预期的追问，按频率排：judge 用什么模型（比被评估的强一档，且必须异源）；golden set 从哪来（先从线上捞一批人工标注，再逐次把事故补进去）；这套东西自己出问题怎么办（面板发现 rubric 混版直接拒绝聚合，而不是给一个没含义的平均分）；多久能上线（第二层一周、第一层两周、第三层一个月，因为它依赖前两层）。

  How to reason about it · think before answering

  1. Do not draw an architecture diagram yet. The trap is that this sounds like build monitoring, so many candidates open with Prometheus and Grafana — that answers infrastructure, not this question. Spend three to five minutes on four things: how often prompts change and how they ship (weekly cadence, canary, rollback); how problems surface today (user complaints, or someone happening to notice); what history exists (how long conversations are retained, whether they can be replayed); and who consumes this (engineers debugging, or an executive watching spend). All four materially change the design, so asking them scores.
  2. Then the trunk, in one sentence: one dataset, two readings. Instrument once, as spans; read across for a single request's call tree (debugging) and stack them for a dashboard (trends and cost). This is the foundation — two data sources will eventually disagree and then nobody trusts either. Many candidates fork here into a monitoring system and an evaluation system, which is the source of every later problem.
  3. Then three layers. Layer one, offline regression: a small stable golden set (15 to 50), covering three things — every route exercised, one item per failure mode (low-confidence fallback, tool budget exhaustion, downstream outage), and the cases behind real past incidents. Each item declares its expected route and a checklist of required facts. The maintenance rule is add, never edit: changing an expectation voids all historical scores. Score with an LLM-as-judge using a different model, and version the rubric, storing that version on every record. This layer runs in CI on every prompt change and emits a number comparable to last time.
  4. Layer two, online observability: every request writes a span tree recording the routing rationale (a model decision, lost forever if not captured), per-node tokens and latency, and degradation and fallback events. The dashboard answers four questions: how much is wrong, where it is slow, what it cost, and which role spent it — that last one is multi-agent specific and the most actionable.
  5. Layer three, online sampled evaluation: fifteen offline cases cannot cover the real traffic distribution, so sample a fraction of live requests (say 1%) through the same judge to get a true quality curve. This layer bridges the other two: offline tells you whether you broke something known, online tells you what real users encountered.
  6. Bring numbers on cost, which is what separates levels. A multi-agent request can produce five to ten model calls, so per-call price is an order of magnitude below the real unit cost and you must price per request. Give the arithmetic: 10k DAU at three sessions each and five calls per session is 150k calls a day; at 2000 input and 500 output tokens, $0.15 and $0.60 per million, that is roughly $90 a day. That number implies two things: per-node attribution shows where to optimise, and evaluation's own cost must be tracked separately, since judge calls are the same order as the system itself and decide whether you evaluate per commit or nightly.
  7. Close on adoption, which many candidates omit: wire evaluation into the release process (block a deploy when pass rate drops below threshold), keep the rubric and golden set in the repository under code review, and pair every mechanism with a failure mode — judges favour same-family models, golden sets get gamed (someone tunes prompts to make it green, and at that moment it is worthless), sampling misses the long tail. A proposal with no stated failure modes reads as book knowledge.
  8. Expect, by frequency: which model judges (one tier above the system under test, and necessarily a different family); where the golden set comes from (start with human-labelled production samples, then append every incident); what happens when this system itself misbehaves (the dashboard refuses to aggregate mixed rubric versions rather than emitting a meaningless average); and how long to build (layer two in a week, layer one in two, layer three in a month since it depends on both).

  答题要点

  • 先用三到五分钟问清四件事：改提示词的频率与发布方式、现在问题怎么被发现、有无历史数据可回放、这套东西给谁看
  • 主干是「一份数据、两种读法」：埋点只做一套 span，横着读是调用树、竖着堆是面板；两套数据源迟早对不上
  • 第一层离线回归：小而稳的 golden set，三层覆盖（每条路由、每种失败模式、历史事故），只增不改，挂 CI
  • 第二层在线观测：span 树记路由理由、每节点 token 与耗时、降级兜底事件；面板回答错了多少/慢在哪/花了多少/钱花在哪个角色
  • 第三层在线采样评估：按比例采样线上请求跑同一套 judge，补上离线覆盖不到的真实分布
  • 成本必须按请求算而非按调用：一次请求 5 到 10 次调用，给出日活一万约 90 美元一天的算式；评估自身成本单独记
  • 收在落地：通过率跌破阈值挡发布、rubric 与 golden set 进仓库走 review
  • 每个机制配失效模式：judge 偏向同源、golden set 会被针对性优化、采样漏长尾——说不出失效模式等于只是读过

  Key points

  • Spend three to five minutes clarifying four things: prompt change cadence and release process, how problems surface today, what replayable history exists, and who the audience is
  • The trunk is one dataset, two readings: instrument once as spans, read across for a call tree and stack for a dashboard; two sources will disagree
  • Layer one, offline regression: a small stable golden set covering every route, every failure mode and past incidents, add-never-edit, wired into CI
  • Layer two, online observability: span trees recording routing rationale, per-node tokens and latency, degradation events; the dashboard answers wrong/slow/cost/which-role
  • Layer three, sampled online evaluation through the same judge, covering the real distribution the offline set cannot
  • Price per request, not per call: five to ten calls per request, with arithmetic showing ~$90/day at 10k DAU; track evaluation's own cost separately
  • Close on adoption: block releases when pass rate drops, keep rubric and golden set in the repo under review
  • Pair every mechanism with a failure mode: judge self-preference, golden set gaming, sampling missing the tail — omitting these reads as book knowledge