面试题库

14 天用 Agent 搭一条 AI 短剧生产线

D9 并发与配额：多集同时开机，还不能把厂商额度打爆

• 多个任务共用一家厂商的额度，你会怎么设计限流？Many jobs share one vendor's quota. How would you design the rate limiting?

  国内高频海外高频进阶#rate-limiting#concurrency#scheduling

  分析过程 · 先想清楚再作答

  1. 这题的题眼是「共用」两个字。只回答一个令牌桶算答了一半，面试官想听的是你按什么维度分桶、以及桶之外还需要什么。
  2. 先给维度：限流要按「厂商 + 接口类别」分桶，不能全局一个桶。同一家的图像和语音是两个独立的配额池，混在一起会让紧的那个把松的那个也拖住。
  3. 再给部件：一个桶不够，要两个。令牌桶管速率（一分钟发几次，数字是厂商定的），信号量管并发（同一时刻挂着几个，数字是你自己定的用来保护内存和钱包）。只有速率控制的话，快速返回的接口一分钟能发几百次；只有并发控制的话，二十个请求同时在飞会把内存挂满。
  4. 然后是关键的工程细节：拿许可的动作必须是非阻塞的。如果任务先被派出去、再在执行流里等令牌，工作槽会被一批低优先任务占死，优先级就静默失效了。正确结构是调度器派活之前先问闸门要许可，拿不到就跳过它去看下一个候选。
  5. 最后落到取值：并发上限不该按 CPU 核数定，这条线几乎没有本地计算，全在等网络；它该按「一次失败要重跑多少东西」和厂商配额来定。
  6. 可预期的追问是「桶的状态放哪」。单进程放内存就够；多进程要放 Redis，用一个原子脚本取令牌，否则每个进程各限各的，加起来照样超。

  How to reason about it · think before answering

  1. The hinge is the word shared. Naming a token bucket only answers half of it; the interviewer wants your bucketing dimension and what else sits around the bucket.
  2. Dimension first: bucket per vendor and per API family, never one global bucket. Image and speech quotas at the same vendor are separate pools, and merging them lets the tight one throttle the loose one.
  3. Then the parts: one bucket is not enough. A token bucket caps rate (calls per minute, a number the vendor sets); a semaphore caps concurrency (how many are in flight, a number you set to protect memory and spend). Rate alone lets a fast endpoint fire hundreds per minute; concurrency alone lets twenty downloads pile up.
  4. The engineering detail that decides everything: admission must be non-blocking. If a job is dispatched first and then waits for a token inside the worker, low-priority work pins every worker slot and priority silently stops working. Ask the gate before dispatch, and skip to the next candidate when it says no.
  5. Finally the numbers: concurrency should not track CPU cores, since this pipeline is almost all network wait. Derive it from vendor quota and from how much work one failure forces you to redo.
  6. Expected follow-up: where does bucket state live. In-memory is fine for one process; across processes it belongs in Redis behind an atomic token-take script, or each process limits itself and the sum still blows the quota.

  答题要点

  • 按「厂商 + 接口类别」分桶，一家的图像和语音各一道闸门。
  • 每道闸门两个部件：令牌桶控速率（厂商给的 RPM），信号量控并发（自己定的在飞上限）。
  • 准入必须非阻塞，拿不到许可就把任务留在队列里，绝不占着工作槽干等。
  • 先抢并发票再取令牌，顺序反了会白白扣掉配额。
  • 多进程部署时桶的状态要外置到 Redis，用原子操作取令牌。

  Key points

  • Bucket per vendor and per API family; image and speech at one vendor get separate gates.
  • Each gate has two parts: a token bucket for rate (the vendor's RPM) and a semaphore for in-flight concurrency (your own number).
  • Admission is non-blocking: if the gate says no, the job stays queued instead of holding a worker slot.
  • Take the concurrency slot before the token, or a rejected admission silently burns quota.
  • Across processes, move bucket state to Redis and take tokens atomically.
• 优先级队列容易出现饿死，你会怎么防？Priority queues starve low-priority work. How do you prevent that?

  国内高频海外高频基础#scheduling#priority-queue#fairness

  分析过程 · 先想清楚再作答

  1. 这题是送分题，但很多人只答一个「老化」就停了，拿不到区分度。区分度在两个补充条件上。
  2. 先说机制：老化，也就是等待越久有效优先级越高，每等过一个阈值就升一档。同档内按入队时间先来先服务。
  3. 第一个补充条件是升档要封顶，而且不许升进最高那一档。否则跑上半小时，队列里全是最高优先级，这一档就名存实亡了。本课的口径是低优先最多升到普通，最高档只留给人工插队。
  4. 第二个补充条件更容易被忽略：如果任务被派出去之后才开始等资源，优先级会静默失效——工作槽被一批低优先任务占着等资源，高优先任务连被取走的机会都没有。所以准入要在调度之前完成。
  5. 结论里要给出可观测量：按优先级统计平均等待与最长等待，再加一个升档次数。这两组数字能直接告诉你老化阈值配得对不对。
  6. 可预期的追问是「除了老化还有别的办法吗」。有：给低优先级预留一部分固定配额（比如每四次调度必须让一个低优先的过），这是加权公平调度的思路，比老化更可控但实现更啰嗦。

  How to reason about it · think before answering

  1. This is the easy one, and most candidates stop after saying aging. The signal is in the two conditions they forget to attach.
  2. The mechanism first: aging, where effective priority rises with waiting time, one step per threshold crossed, with first-in-first-out inside a tier.
  3. Condition one: cap the promotion, and never let it reach the top tier. Otherwise after half an hour every queued job is top priority and the tier means nothing. Our rule is that low may rise to normal, and the top tier stays reserved for human escalation.
  4. Condition two is the one people miss: if a job waits for resources after dispatch, priority silently stops working, because worker slots are pinned by low-priority jobs waiting on quota and the urgent job is never picked up. Admission must happen before dispatch.
  5. Close with the observable: track average and maximum wait per tier plus a promotion counter. Those two numbers tell you directly whether the aging threshold is right.
  6. Expected follow-up: alternatives to aging. Reserved shares work too, where every fourth dispatch must go to a low-priority job. That is weighted fair queuing, more controllable but noisier to implement.

  答题要点

  • 用老化：等待时间越长有效优先级越高，同档内先来先服务。
  • 升档要封顶，绝不能升进最高那一档，否则最高档形同虚设。
  • 准入要放在调度之前，否则工作槽被低优先任务占着等资源，优先级会静默失效。
  • 按优先级统计平均等待、最长等待与升档次数，用它来校准老化阈值。
  • 备选方案是给低优先级预留固定份额的加权公平调度，比老化更可控但实现更复杂。

  Key points

  • Use aging: effective priority rises with wait time, first-in-first-out within a tier.
  • Cap promotion and never let it reach the top tier, or the top tier stops meaning anything.
  • Admit before dispatch, otherwise worker slots pinned on quota make priority silently useless.
  • Track per-tier average wait, max wait and promotion count, and tune the aging threshold from those.
  • The alternative is weighted fair queuing with a reserved share for low priority: more controllable, more code.