面试题库

30 天从前端工程师到 Agent 工程师

D4 模型接入与系统提示词：多 provider 抽象与 fallback、覆盖默认人设（dg P03/P04/M04）

• 为什么生产级 Agent 通常要接入多个模型 provider？Why do production agents usually integrate more than one model provider?

  国内高频海外高频基础#model-routing#reliability

  分析过程 · 先想清楚再作答

  1. 先判断这题问的是「可用性」还是「架构」。只答「防止挂掉」拿不到分，因为面试官想看的是你有没有真的算过账、踩过坑。
  2. 从一条因果链推：模型 API 是外部依赖 → 外部依赖必然有故障率 → 你的可用性上限被它锁死 → 所以要么接受这个上限，要么加冗余。
  3. 把可用性说成数字才有说服力：单家 99.5% 意味着每月约 3.6 小时不可用；三家独立故障时理论不可用时间降到秒级。数量级差异比形容词有力得多。
  4. 第二个理由往往被忽略，但更能体现工程视角：模型的价格和能力每月都在变，接入成本高会让你因为「改起来麻烦」而一直用贵的慢的那个——高耦合真正的代价是剥夺未来的选择权。
  5. 这里有个必须自己先说破的前提：那个数量级是拿「三家故障互不相关」算出来的。如果三家其实都走同一个聚合网关、共用同一把 key（很多人的第一版就是这样），网关一挂三家一起挂，冗余是假的，聚合网关反而成了新的单点。真正的独立要落到不同厂商的直连端点、各自的凭证和计费上。主动点破这一条，比背出 0.005 的三次方更能体现你真的部署过。
  6. 可以预期的追问：多接几家不是更贵吗？答案是不会——正常路径只调一家，多的只是配置和一层抽象；真正贵的是 fallback 被频繁触发，那说明你该查主 provider 而不是砍掉冗余。

  How to reason about it · think before answering

  1. First decide whether this is an availability question or an architecture question; answering only 'so it doesn't go down' reads as inexperienced.
  2. Follow the causal chain: the model API is an external dependency, dependencies have failure rates, your ceiling is capped by theirs, so you either accept the cap or add redundancy.
  3. Quantify it: 99.5% monthly availability is about 3.6 hours of downtime; three independently failing providers push that to seconds. Orders of magnitude beat adjectives.
  4. The second reason shows engineering maturity: model pricing and capability shift monthly, and high switching cost means you stay on the expensive slow one out of inertia — coupling really costs you future optionality.
  5. Say the premise out loud before they ask: that order of magnitude assumes the three providers fail independently. If all three are model ids behind one aggregator gateway on a single key — which is what most first versions look like — the gateway going down takes all three with it, the redundancy is fake, and the aggregator has become the new single point of failure. Real independence means direct endpoints at different vendors, with separate credentials and billing. Naming this yourself signals operational experience far more than reciting 0.005 cubed.
  6. Expect the follow-up: isn't this more expensive? No — the happy path calls one provider; what costs money is fallback firing often, which is a signal to investigate the primary, not to remove redundancy.

  答题要点

  • 模型 API 是外部依赖，厂商故障、限流、模型下线都是每月都会遇到的日常，不是小概率事件
  • 单家 99.5% 可用性等于每月约 3.6 小时不可用；多家冗余能把理论不可用时间降低几个数量级
  • 价格与能力每月都在变，统一抽象层让换模型变成改配置，保住了未来做选择的自由
  • 正常路径只调一家，冗余的成本是一层抽象而不是多倍账单

  Key points

  • The model API is an external dependency; outages, rate limits and model deprecations are monthly realities
  • 99.5% monthly availability is roughly 3.6 hours down; multi-provider redundancy cuts that by orders of magnitude
  • Pricing and capability shift constantly, so an abstraction layer turns model swaps into config changes
  • The happy path still calls one provider — redundancy costs an abstraction, not a multiplied bill
• 设计模型 fallback 策略时要权衡哪些因素？What trade-offs shape a model fallback strategy?

  国内高频海外高频进阶#model-routing#reliability#cost

  分析过程 · 先想清楚再作答

  1. 题眼在「权衡」两个字——面试官不要你背一个 for 循环，他要看你知不知道 fallback 是有代价的。
  2. 先拆出第一个关键判断：不是所有错误都该 fallback，而分类的依据不是状态码的首位数字，是「换一家有没有可能变好」。400 请求体不合法、403 被安全策略拦截，换谁都一样，重试只是把同一个 bug 再犯一遍、白花两倍的钱和时间；408 超时、429 限流、5xx 服务端故障是对方的问题，换一家大概率能成。最容易答错的是 402 余额不足和 404 模型被下线或改名——它们同属 4xx、长得像「你的问题」，其实换一家完全可能成功；401 则要看凭证怎么管，三家共用一把网关 key 时换了也没用，各有各的 key 时 A 被吊销切到 B 完全能救。分类是 fallback 的第一步，不是重试。
  3. 再说成本：切换意味着同一段 prompt 你付了两次钱，三家链路最坏是三倍成本。这条一定要主动说出来，它区分了「写过」和「上过线」。
  4. 然后是延迟：串行 fallback 的总耗时是各家超时值的累加。如果每家给 15 秒、三家串下来用户要等 45 秒，那还不如早点失败。所以超时值必须按 provider 分别设，且要设总预算上限。
  5. 最后是雪崩，这是最容易被追问的点：主 provider 限流时你把全部流量瞬间压到备用上，很可能把备用也压垮。所以要加熔断——连续失败 N 次就暂时摘掉该 provider，过一段时间放少量流量试探。
  6. 可以预期的追问：怎么知道该摘多久？答案是指数退避 + 半开状态试探，和数据库连接池的熔断是同一套思路。

  How to reason about it · think before answering

  1. The word 'trade-offs' is the hinge: they are not asking for a for-loop, they want to know you understand fallback has costs.
  2. First key judgment: not every error deserves a fallback, and the test is not the leading digit of the status code but whether another provider could plausibly succeed. A 400 (malformed body) or 403 (blocked by safety policy) fails everywhere, so retrying repeats your own bug at double the cost and latency; a 408, 429 or 5xx is theirs and usually succeeds elsewhere. The two that people get wrong are 402 (out of credit) and 404 (model retired or renamed): both are 4xx, both look like your fault, and both are fixed by switching. A 401 depends on how credentials are managed — one shared gateway key fails everywhere, but per-provider keys mean a revoked key on A is survivable on B. Classification comes before retry.
  3. Cost: switching means paying for the same prompt twice, up to 3x across a three-provider chain. Volunteering this separates people who shipped from people who only read about it.
  4. Latency: serial fallback accumulates timeouts. Three providers at 15s each means a 45s wait — worse than failing fast. Timeouts must be per-provider with an overall budget.
  5. Thundering herd is the most common follow-up: when the primary rate-limits, shifting all traffic at once can take down the backup too. Hence circuit breaking — drop a provider after N consecutive failures, then probe with a trickle.
  6. Expect: how long do you drop it for? Exponential backoff with a half-open probe — the same pattern as database connection pool breakers.

  答题要点

  • 先分类再重试，判据是「换一家有没有可能变好」而不是状态码首位：400/403 不该切，408/429/5xx 该切，402 余额不足和 404 模型下线同样该切，401 取决于三家是否共用同一把凭证
  • 成本：每次 fallback 都要重付一遍 prompt 的钱，链路越长最坏成本越高
  • 延迟：串行 fallback 的耗时是各超时值累加，必须按 provider 分设超时并设总预算
  • 雪崩防护：主 provider 故障时全量流量压向备用会把备用也压垮，需要熔断 + 指数退避 + 半开试探

  Key points

  • Classify before retrying, judging by whether another provider could plausibly succeed rather than the leading digit: 400/403 must not fail over; 408/429/5xx should; so should 402 (out of credit) and 404 (model retired); 401 depends on whether the providers share one key
  • Cost: every fallback re-pays for the same prompt, so worst-case cost scales with chain length
  • Latency: serial fallback sums the timeouts, so set per-provider timeouts plus an overall budget
  • Thundering herd: shifting full traffic to the backup can topple it too — use circuit breaking with exponential backoff and half-open probes
• 如何在成本和延迟之间给不同任务选择合适的模型？How do you pick the right model per task, balancing cost against latency?

  国内高频海外高频进阶#model-routing#cost#latency

  分析过程 · 先想清楚再作答

  1. 这题考的是「你有没有真的在花自己的钱」。答「用最好的模型」是最差的答案，答「按需选择」太空，要给出可执行的分档维度。
  2. 先建立核心事实：不同模型的价格能差 50 倍以上，而你的任务里很大一部分根本不需要最强的模型。用旗舰模型做意图识别，等于开跑车去楼下取快递。
  3. 然后给出三个可操作的路由维度：任务类型（分类抽取走便宜模型，长文推理走强模型）、延迟要求（前台用户在等就走低延迟，后台批处理可以慢而便宜）、输入长度（超长上下文只有部分模型支持且价格陡增）。
  4. 结论要落到数字上才有说服力：1 万轮对话每轮 2000 token，全走旗舰约 300 元一天；把六成粗活改走小模型后降到 125 元左右，一年省六万多，用户感知不到差别。
  5. 还要主动说出实现上的取舍：先按任务类型静态分档，不要一上来就做「让模型判断该用哪个模型」的动态路由——那个方案本身又要多一次模型调用，延迟和成本可能得不偿失，等有真实数据再优化。
  6. 可以预期的追问：怎么验证降档没有损失质量？答案是准备 golden set，对同一批输入跑两档模型，用人工或 LLM-as-judge 比对准确率，把降档决策建立在数据上而不是感觉上。

  How to reason about it · think before answering

  1. This question tests whether you have ever spent your own money. 'Use the best model' is the worst answer; 'it depends' is too vague — give actionable routing dimensions.
  2. Establish the core fact: model pricing spans 50x or more, and much of your workload does not need the strongest model. Using a flagship for intent detection is driving a sports car to fetch a parcel downstairs.
  3. Give three routing dimensions: task type (classification and extraction go cheap, long-form reasoning goes strong), latency requirement (foreground users need low latency, background batches can be slow and cheap), and input length (only some models handle very long context, and pricing rises steeply).
  4. Quantify it: 10k conversations a day at 2000 tokens each costs roughly 300 CNY/day on a flagship; routing the 60% of grunt work to a small model drops it to about 125 CNY/day, saving 60k+ CNY a year with no perceptible quality change.
  5. Volunteer the implementation trade-off: start with static tiers by task type. Dynamic routing that asks a model which model to use adds another model call, and the latency and cost may not pay for themselves — optimize once you have real data.
  6. Expect: how do you verify the cheaper tier did not hurt quality? A golden set — run both tiers over the same inputs and compare with human or LLM-as-judge scoring, so the decision rests on data rather than vibes.

  答题要点

  • 不同模型价格能差 50 倍以上，用旗舰模型做意图识别是明显的浪费
  • 三个路由维度：任务类型（分类抽取 vs 推理生成）、延迟要求（前台 vs 后台）、输入长度（是否需要超长上下文）
  • 实现上给调用层加 tier 参数，按任务静态分档挑起始 provider，fallback 逻辑完全复用
  • 先静态分档再考虑动态路由，让模型判断该用哪个模型本身要多一次调用，可能得不偿失
  • 用 golden set 对比两档模型的准确率，把降档决策建立在数据上

  Key points

  • Model pricing spans 50x or more, so a flagship doing intent detection is obvious waste
  • Three routing dimensions: task type, latency requirement, and input length
  • Add a tier parameter to the call layer, pick the starting provider statically, and reuse the fallback chain
  • Prefer static tiers first — dynamic model-picks-model routing adds a call and may not pay off
  • Validate downgrades against a golden set rather than intuition