面试题库

30 天从前端工程师到 Agent 工程师

D16 Supervisor 动态路由：structured output 路由、override、routingReason

• 为什么要让模型输出 structured output 而不是自然语言来做路由？自然语言到底差在哪？Why use structured output rather than natural language for routing? What exactly goes wrong with free text?

  国内高频海外高频进阶#structured-output#routing#reliability

  分析过程 · 先想清楚再作答

  1. 这题最容易答成「结构化更规范、更好解析」——这是形容词，不是理由。面试官想听的是一个具体的失败场景，最好是你真的调过的那种。
  2. 把最锋利的一刀先亮出来：**自然语言路由的失败是静默的**。模型回「我觉得这个可以让查订单的同事看一下」，它其实判对了，但说的是人话不是 id，正则匹配不上就落进兜底，日志里只留下一个 smalltalk。模型是对的、解析是错的，而它和「模型判错了」在日志里长得一模一样。你会去调提示词，调两天才发现问题在那三行正则。
  3. 然后给三个漏洞，一条一条对上结构化输出解决了什么：输出会漂移（今天回「订单查询」明天回「查订单」，正则永远追不上，模型小版本升级你就掉准确率）；没有置信度（自然语言里没有「我有多大把握」这个信息，你没法区分它很确定还是在猜）；拼错或自造的路由名要到运行时才炸（枚举是一道编译期就存在的闸门）。
  4. 接着说清机制，别停在「用 zod 更规范」：把 schema 发进请求（response_format 里的 json_schema），模型的解码过程被枚举约束；回来之后**用同一份声明再校验一遍**。一份声明两用，请求与校验不会漂移。
  5. 关键的反直觉点，答到这里就拉开差距了：**结构化输出不等于不用校验**。不是所有网关、所有模型都严格执行 schema，降级到备用模型时更说不准。所以解析函数的返回类型应该是「一段待校验的东西」，而不是「已经是 RouteDecision」。
  6. 可以预期的追问：那不支持 json_schema 的模型怎么办？答案是退回「few-shot 加严格提示词加自己校验」，闸门仍然在你的枚举校验那一步——真正兜底的从来不是模型的自觉，是你的解析层。

  How to reason about it · think before answering

  1. The trap is answering structured output is cleaner and easier to parse. Those are adjectives, not reasons. The interviewer wants a concrete failure you have actually debugged.
  2. Lead with the sharpest point: free-text routing fails silently. The model replies I think the order desk should look at this — it judged correctly, but it spoke prose, not an id. Your regex misses, you fall through to the default, and the log shows only smalltalk. A correct model with a broken parser looks exactly like a wrong model, so you spend two days tuning a prompt that was never the problem.
  3. Then list three holes and map each to what structured output fixes: wording drifts across versions so regexes never catch up; there is no confidence signal, so you cannot tell certainty from guessing; and an invented route name only explodes at runtime, whereas an enum is a gate that exists before the request is even sent.
  4. Explain the mechanism rather than stopping at zod is nicer: send the schema in the request (response_format with a json_schema), so decoding is constrained by the enum, then validate the response with the same declaration. One declaration used twice means request and validation cannot drift apart.
  5. The counterintuitive point that separates candidates: structured output does not remove the need to validate. Not every gateway or model enforces the schema strictly, and a fallback model may not at all. Your parse function should return something-to-be-validated, not an already-typed decision.
  6. Expect: what if the model does not support json_schema? Fall back to few-shot plus a strict prompt plus your own validation. The real gate was never the model's discipline; it is your parsing layer.

  答题要点

  • 自然语言路由的失败是静默的：模型判对了但说的是人话，正则匹配不上就落兜底，和判错在日志里完全一样
  • 三个漏洞：措辞会漂移（正则追不上）、没有置信度（分不清确定与猜）、自造的路由名要到运行时才炸
  • 机制是一份声明两用：schema 随请求发出去约束解码，回来后用同一份声明校验，请求与校验不会漂移
  • 枚举是编译期就存在的闸门，把「拼错的路由名」从线上事故降级成一次解析失败
  • 结构化输出不等于不用校验：网关和降级模型未必严格执行 schema，解析函数的返回类型应该是「待校验」而不是「已经是」

  Key points

  • Free-text routing fails silently: a correct judgement in prose misses your regex and falls through, looking identical to a wrong judgement in the logs
  • Three holes: wording drifts, there is no confidence signal, and invented route names only fail at runtime
  • One declaration used twice: the schema constrains decoding in the request and validates the response, so the two cannot drift
  • An enum is a gate that exists before the call, turning a misspelled route from an incident into a parse failure
  • Structured output does not remove validation — gateways and fallback models may not enforce the schema, so parsing must return an unvalidated value
• routingReason 这类调试信息在生产系统里有什么价值？只是打日志而已吗？What is a field like routingReason actually worth in production? Is it just logging?

  国内高频海外高频进阶#observability#routing#debugging

  分析过程 · 先想清楚再作答

  1. 这题看着像水题，其实在筛「有没有真的排查过线上问题」。答「方便调试」就结束的人，基本没值过班。
  2. 先给一条不可回避的事实：**路由决策是模型做的，而模型不可复现**。同一句话下次未必给同样的判断，你没法重跑一遍去看「当时是怎么想的」。所以理由必须在当时就写下来，否则那次判断永远丢了。这一条把 routingReason 从「日志」抬到了「唯一的审计证据」。
  3. 然后给三个具体用途，每个都要能落地：一是把「模型判错了」和「解析或兜底出错了」分开，前缀写成 fallback 加原因码，一眼就能分辨；二是攒下一版提示词的素材，把一周内落进兜底的请求按原因分组，会看到集中的几类意图缺描述；三是它是离线评估的输入——标准样本集要评的不只是最终回答，还有分诊准不准，而这件事只有当时记了判断和理由才评得了。
  4. 写法上有个细节值得主动说：**结构化的壳加自然语言的芯**。前缀（fallback 加原因、override 加目标）用来聚合统计，后面那句人话用来看具体这一单。整条都写成自然语言，就退回成本章批判的那种东西了。
  5. 再补一条容易被忽略的：兜底和人工改派都不要擦掉模型的原判，原样拼进理由里。否则一周后没人说得清这一单是模型判错了还是本来就被人改过——多写二十个字符，省掉一次翻遍代码的排查。
  6. 可以预期的追问：这些字段会不会带来隐私或成本问题？答案是会，所以理由里只写判断依据不写用户原文，长度设上限（比如 120 字），并且和链路追踪共用同一个 run 标识，别另起一套。

  How to reason about it · think before answering

  1. This looks like a throwaway question but it screens for whether you have ever been on call. Anyone who stops at it helps with debugging has not.
  2. Start with the fact you cannot design around: the routing decision is made by a model, and models are not reproducible. The same sentence may be judged differently next time, so you cannot re-run to see what it was thinking. The reason must be captured at decision time or it is gone forever — that is what turns this field from a log line into the only audit evidence you have.
  3. Then give three concrete uses. One, it separates a wrong model judgement from a parsing or fallback problem, provided the prefix carries a cause code. Two, it is raw material for the next prompt revision: group a week of fallbacks by cause and the missing intent descriptions jump out. Three, it feeds offline evaluation — a golden set should score routing accuracy, not just the final answer, and that is only scorable if the decision and its reason were recorded.
  4. Mention the shape: a structured prefix wrapping a human sentence. The prefix (fallback plus cause, override plus target) is what you aggregate on; the sentence is what you read for one specific case. Making the whole field prose puts you right back in the failure mode this chapter argues against.
  5. Add the detail people skip: neither a fallback nor a human override should erase the model's original judgement — carry it into the reason. Otherwise nobody can later tell whether the model got it wrong or a human redirected it. Twenty extra characters save an afternoon of archaeology.
  6. Expect: do these fields create privacy or cost problems? Yes, so record the basis for the decision rather than the user's raw text, cap the length, and reuse the same run identifier as your tracing instead of inventing a parallel one.

  答题要点

  • 路由决策由模型做出且不可复现，理由必须在当时写下来，否则那次判断永远丢了——它是唯一的审计证据
  • 用途一：把「模型判错」和「解析或兜底出错」分开，靠 fallback 加原因码一眼分辨
  • 用途二：把一周内落进兜底的请求按原因分组，直接得到下一版分诊提示词该补什么
  • 用途三：它是离线评估的输入，分诊准确率这个指标只有记了当时的判断与理由才评得了
  • 写法是结构化的壳加自然语言的芯：前缀用于聚合统计，人话用于看具体这一单
  • 兜底与人工改派都要保留模型原判；理由只写判断依据不写用户原文，长度设上限，并复用链路追踪的 run 标识

  Key points

  • The decision comes from a model and is not reproducible, so the reason must be captured at decision time — it is the only audit evidence you get
  • Use one: it separates a wrong model judgement from a parsing or fallback failure, via a cause code in the prefix
  • Use two: grouping a week of fallbacks by cause tells you exactly what the next routing prompt is missing
  • Use three: it feeds offline evaluation, since routing accuracy can only be scored if the decision and reason were recorded
  • Shape it as a structured prefix around a human sentence: aggregate on the prefix, read the sentence for one case
  • Keep the model's original judgement through fallbacks and overrides; store the basis rather than raw user text, cap the length, and reuse the tracing run id