面试题库

30 天从前端工程师到 Agent 工程师

D4 模型接入与系统提示词：多 provider 抽象与 fallback、覆盖默认人设（dg P03/P04/M04）

• 如何在成本和延迟之间给不同任务选择合适的模型？How do you pick the right model per task, balancing cost against latency?

  国内高频海外高频进阶#model-routing#cost#latency

  分析过程 · 先想清楚再作答

  1. 这题考的是「你有没有真的在花自己的钱」。答「用最好的模型」是最差的答案，答「按需选择」太空，要给出可执行的分档维度。
  2. 先建立核心事实：不同模型的价格能差 50 倍以上，而你的任务里很大一部分根本不需要最强的模型。用旗舰模型做意图识别，等于开跑车去楼下取快递。
  3. 然后给出三个可操作的路由维度：任务类型（分类抽取走便宜模型，长文推理走强模型）、延迟要求（前台用户在等就走低延迟，后台批处理可以慢而便宜）、输入长度（超长上下文只有部分模型支持且价格陡增）。
  4. 结论要落到数字上才有说服力：1 万轮对话每轮 2000 token，全走旗舰约 300 元一天；把六成粗活改走小模型后降到 125 元左右，一年省六万多，用户感知不到差别。
  5. 还要主动说出实现上的取舍：先按任务类型静态分档，不要一上来就做「让模型判断该用哪个模型」的动态路由——那个方案本身又要多一次模型调用，延迟和成本可能得不偿失，等有真实数据再优化。
  6. 可以预期的追问：怎么验证降档没有损失质量？答案是准备 golden set，对同一批输入跑两档模型，用人工或 LLM-as-judge 比对准确率，把降档决策建立在数据上而不是感觉上。

  How to reason about it · think before answering

  1. This question tests whether you have ever spent your own money. 'Use the best model' is the worst answer; 'it depends' is too vague — give actionable routing dimensions.
  2. Establish the core fact: model pricing spans 50x or more, and much of your workload does not need the strongest model. Using a flagship for intent detection is driving a sports car to fetch a parcel downstairs.
  3. Give three routing dimensions: task type (classification and extraction go cheap, long-form reasoning goes strong), latency requirement (foreground users need low latency, background batches can be slow and cheap), and input length (only some models handle very long context, and pricing rises steeply).
  4. Quantify it: 10k conversations a day at 2000 tokens each costs roughly 300 CNY/day on a flagship; routing the 60% of grunt work to a small model drops it to about 125 CNY/day, saving 60k+ CNY a year with no perceptible quality change.
  5. Volunteer the implementation trade-off: start with static tiers by task type. Dynamic routing that asks a model which model to use adds another model call, and the latency and cost may not pay for themselves — optimize once you have real data.
  6. Expect: how do you verify the cheaper tier did not hurt quality? A golden set — run both tiers over the same inputs and compare with human or LLM-as-judge scoring, so the decision rests on data rather than vibes.

  答题要点

  • 不同模型价格能差 50 倍以上，用旗舰模型做意图识别是明显的浪费
  • 三个路由维度：任务类型（分类抽取 vs 推理生成）、延迟要求（前台 vs 后台）、输入长度（是否需要超长上下文）
  • 实现上给调用层加 tier 参数，按任务静态分档挑起始 provider，fallback 逻辑完全复用
  • 先静态分档再考虑动态路由，让模型判断该用哪个模型本身要多一次调用，可能得不偿失
  • 用 golden set 对比两档模型的准确率，把降档决策建立在数据上

  Key points

  • Model pricing spans 50x or more, so a flagship doing intent detection is obvious waste
  • Three routing dimensions: task type, latency requirement, and input length
  • Add a tier parameter to the call layer, pick the starting provider statically, and reuse the fallback chain
  • Prefer static tiers first — dynamic model-picks-model routing adds a call and may not pay off
  • Validate downgrades against a golden set rather than intuition