面试题库

30 天从前端工程师到 Agent 工程师

D13 cron 调度（中心调度→stream 投递）+ 成本计量（token→USD 台账、usage report）

• 一份 LLM 应用的 usage report 通常要覆盖哪些维度？这些维度分别用来做什么决策？Which dimensions should a usage report for an LLM product cover, and what decision does each one drive?

  国内高频海外高频进阶#observability#cost#reporting

  分析过程 · 先想清楚再作答

  1. 这题最容易答成罗列维度：按用户、按天、按模型、按功能……列得越全越显得没想过。区分度在后半句——每个维度对应的是哪一类行动。列不出行动，说明你只做过报表没用过报表。
  2. 先给三个主维度和它们各自的行动类型：按用户切是商业动作（谁该涨价、谁在滥用、定价分层能不能覆盖成本）；按天切是排障动作（对齐发布时间线，找出是哪次上线让成本跳了台阶）；按模型和调用类型切是优化动作（验证分层路由有没有真省到钱、embedding 的量是不是失控了）。三个维度对应三个不同的看板受众。
  3. 然后升一层，指出绝对金额没有信息量，真正有用的是带分母的单位经济学指标：每次执行成本（当月总成本除以 run 数）、每用户月成本（除以活跃用户数）、每美元产出（完成的业务动作数除以总成本）。前两个用来判断定价能不能覆盖成本，第三个用来判断这套系统值不值得继续投入。
  4. 举一个能落地的用法证明你真用过：每次执行成本这个比值是把尺子。如果用户数没涨而单次成本涨了，几乎一定是某次上线让提示词变长了，或者某个工具的返回体膨胀了——这个信号通常比超时告警早好几天出现，所以成熟团队会把成本曲线和错误率、延迟并排挂在值班大盘上。
  5. 最后补一个大多数人会漏的维度：失败与降级。失败的调用照样收费，fallback 会让一次业务操作跨两三次收费调用。报表里不单独切出这一块，你和厂商账单的差额就会恰好集中在故障期，也就是最需要看清成本的时候。
  6. 可以预期的追问：报表要做到什么实时度？答案是分层——按天的汇总离线跑就够，但配额和预算护栏需要近实时的当月累计，通常用一张按用户按月的汇总表增量更新，而不是每次请求都扫一遍明细。

  How to reason about it · think before answering

  1. The trap is listing dimensions: by user, by day, by model, by feature. Length signals you have not thought about it. The discriminator is the second half — which action each dimension drives. No action means you built reports but never used one.
  2. Give three primary dimensions with their action type: by user is a commercial action (who to reprice, who is abusing, whether tiering covers cost); by day is a debugging action (align with the release timeline to find which deploy stepped the cost up); by model and call kind is an optimization action (did tiered routing actually save money, is embedding volume running away). Three dimensions, three different dashboard audiences.
  3. Then go up a level: absolute dollars carry no information. What matters are unit-economics ratios with a denominator — cost per run (monthly cost over run count), cost per active user per month, and business actions completed per dollar. The first two say whether pricing covers cost; the third says whether the system deserves further investment.
  4. Prove you have used it with a concrete pattern: cost per run is a ruler. If user count is flat but cost per run climbs, it is almost always a deploy that lengthened the prompt or a tool whose response body grew. That signal usually appears days before latency alerts, which is why mature teams put the cost curve next to error rate and latency on the on-call dashboard.
  5. Add the dimension most people miss: failures and fallbacks. Failed calls are still billed, and a fallback spans two or three billable calls per business operation. Without slicing that out, your gap against the vendor invoice concentrates exactly during incidents, when you most need cost clarity.
  6. Expect: how fresh does the report need to be? Tier it — daily rollups can run offline, but quota and budget guardrails need near-real-time month-to-date totals, usually from an incrementally updated per-user monthly summary table rather than scanning the detail rows on every request.

  答题要点

  • 按用户切是商业动作（定价分层、异常账号），按天切是排障动作（对齐发布找成本跳变），按模型和调用类型切是优化动作（验证分层路由、盯 embedding 用量）
  • 绝对金额没有信息量，要看带分母的指标：每次执行成本、每用户月成本、每美元产出
  • 每次执行成本是把尺子：用户数没涨而单次成本涨了，通常是提示词变长或工具返回体膨胀，比超时告警早好几天出现
  • 必须单独切出失败与降级的开销，否则和厂商账单的差额会集中在故障期
  • 实时度要分层：按天汇总可离线跑，预算护栏需要近实时的当月累计，用增量汇总表而不是每次扫明细

  Key points

  • By user drives commercial decisions, by day drives debugging, and by model or call kind drives optimization — three dimensions, three audiences
  • Absolute dollars say nothing; use ratios with a denominator: cost per run, cost per active user per month, and business actions per dollar
  • Cost per run is a ruler: flat users with rising per-run cost usually means a longer prompt or a bloated tool response, and it shows days before latency alerts
  • Slice out failed and fallback calls, or your gap against the vendor invoice concentrates during incidents
  • Tier the freshness: daily rollups offline, near-real-time month-to-date totals from an incremental summary table for budget guardrails