面试题库

14 天用 Agent 搭一条 AI 短剧生产线

D9 并发与配额：多集同时开机，还不能把厂商额度打爆

• 多个任务共用一家厂商的额度，你会怎么设计限流？Many jobs share one vendor's quota. How would you design the rate limiting?

  国内高频海外高频进阶#rate-limiting#concurrency#scheduling

  分析过程 · 先想清楚再作答

  1. 这题的题眼是「共用」两个字。只回答一个令牌桶算答了一半，面试官想听的是你按什么维度分桶、以及桶之外还需要什么。
  2. 先给维度：限流要按「厂商 + 接口类别」分桶，不能全局一个桶。同一家的图像和语音是两个独立的配额池，混在一起会让紧的那个把松的那个也拖住。
  3. 再给部件：一个桶不够，要两个。令牌桶管速率（一分钟发几次，数字是厂商定的），信号量管并发（同一时刻挂着几个，数字是你自己定的用来保护内存和钱包）。只有速率控制的话，快速返回的接口一分钟能发几百次；只有并发控制的话，二十个请求同时在飞会把内存挂满。
  4. 然后是关键的工程细节：拿许可的动作必须是非阻塞的。如果任务先被派出去、再在执行流里等令牌，工作槽会被一批低优先任务占死，优先级就静默失效了。正确结构是调度器派活之前先问闸门要许可，拿不到就跳过它去看下一个候选。
  5. 最后落到取值：并发上限不该按 CPU 核数定，这条线几乎没有本地计算，全在等网络；它该按「一次失败要重跑多少东西」和厂商配额来定。
  6. 可预期的追问是「桶的状态放哪」。单进程放内存就够；多进程要放 Redis，用一个原子脚本取令牌，否则每个进程各限各的，加起来照样超。

  How to reason about it · think before answering

  1. The hinge is the word shared. Naming a token bucket only answers half of it; the interviewer wants your bucketing dimension and what else sits around the bucket.
  2. Dimension first: bucket per vendor and per API family, never one global bucket. Image and speech quotas at the same vendor are separate pools, and merging them lets the tight one throttle the loose one.
  3. Then the parts: one bucket is not enough. A token bucket caps rate (calls per minute, a number the vendor sets); a semaphore caps concurrency (how many are in flight, a number you set to protect memory and spend). Rate alone lets a fast endpoint fire hundreds per minute; concurrency alone lets twenty downloads pile up.
  4. The engineering detail that decides everything: admission must be non-blocking. If a job is dispatched first and then waits for a token inside the worker, low-priority work pins every worker slot and priority silently stops working. Ask the gate before dispatch, and skip to the next candidate when it says no.
  5. Finally the numbers: concurrency should not track CPU cores, since this pipeline is almost all network wait. Derive it from vendor quota and from how much work one failure forces you to redo.
  6. Expected follow-up: where does bucket state live. In-memory is fine for one process; across processes it belongs in Redis behind an atomic token-take script, or each process limits itself and the sum still blows the quota.

  答题要点

  • 按「厂商 + 接口类别」分桶，一家的图像和语音各一道闸门。
  • 每道闸门两个部件：令牌桶控速率（厂商给的 RPM），信号量控并发（自己定的在飞上限）。
  • 准入必须非阻塞，拿不到许可就把任务留在队列里，绝不占着工作槽干等。
  • 先抢并发票再取令牌，顺序反了会白白扣掉配额。
  • 多进程部署时桶的状态要外置到 Redis，用原子操作取令牌。

  Key points

  • Bucket per vendor and per API family; image and speech at one vendor get separate gates.
  • Each gate has two parts: a token bucket for rate (the vendor's RPM) and a semaphore for in-flight concurrency (your own number).
  • Admission is non-blocking: if the gate says no, the job stays queued instead of holding a worker slot.
  • Take the concurrency slot before the token, or a rejected admission silently burns quota.
  • Across processes, move bucket state to Redis and take tokens atomically.
• 收到限流响应之后，除了退避重试还该做什么？Beyond backing off and retrying, what else should happen when you get rate limited?

  国内高频海外高频深入#rate-limiting#error-handling#retry

  分析过程 · 先想清楚再作答

  1. 这题在考你有没有真在生产里被限流打过。只答「指数退避加抖动」是标准答案的前半段，面试官等的是后半段。
  2. 先把限流摆正位置：它不是错误，是信号。它告诉你此刻的发送速率超过了厂商愿意接受的速率。既然是信号，就该有反馈动作，而不只是重试。
  3. 第一个动作是主动降速：把令牌桶罚一档，接下来一两个窗口只发一半令牌。不降速的话，退避结束后你会用同样的速度再撞一次，重试次数越多越糟。
  4. 第二个动作是别在退避里占着执行流。正确做法是把任务重新入队并记一个「不早于」时间戳，槽位立刻还回去给别的任务。
  5. 第三个动作是分类：限流和服务端错误可以重试，鉴权失败、余额不足、参数错误、内容审核不通过一次都不该重试——重试只会让你在一分钟里把同一个错误犯五遍，还白占配额。MiniMax 这边 1002 是限流、1039 是 TPM 维度的限流，1004 鉴权、1008 余额、2013 参数、1026 和 1027 是内容审核。
  6. 第四个动作是把限流次数记进指标。撞得多说明闸门配小了或者配大了，这个数字是你回头调参数的唯一依据。
  7. 可预期的追问是「退避上限怎么定」。定在业务能等的时间上，超过就转降级：换更小的分辨率、更短的时长，或者干脆排到下一批。

  How to reason about it · think before answering

  1. This question separates people who have actually been throttled in production. Exponential backoff with jitter is only the first half of the answer.
  2. Frame it correctly: throttling is a signal, not an error. It says your current send rate exceeds what the vendor will accept right now, so it deserves a feedback action, not just a retry.
  3. Action one is to slow down on purpose: penalize the bucket so the next window or two issues half the tokens. Without that, you finish the backoff and hit the same wall at the same speed.
  4. Action two is to not hold an execution slot while waiting. Requeue the job with a not-before timestamp and hand the slot back immediately.
  5. Action three is classification. Throttling and server errors are retryable; auth failure, insufficient balance, invalid parameters and content-policy rejections are not, and retrying them just repeats one mistake five times while consuming quota. At MiniMax, 1002 is rate limiting and 1039 is the token-per-minute variant, while 1004 is auth, 1008 is balance, 2013 is bad parameters and 1026 or 1027 are content rejections.
  6. Action four is to record throttle counts as a metric. That number is the only evidence you have when you later retune the gate.
  7. Expected follow-up: how to cap the backoff. Cap it at what the business can wait for, then degrade instead of retrying: smaller resolution, shorter duration, or push the job into the next batch.

  答题要点

  • 把限流当信号：退避的同时给令牌桶降档，接下来的窗口只发一半令牌。
  • 退避期间把任务重新入队并记一个不早于时间戳，工作槽立刻还回去。
  • 退避要带抖动，否则同时被限的任务会同时醒来再撞一次。
  • 严格区分可重试与不可重试：鉴权、余额、参数、内容审核一次都不重试。
  • 把限流次数记成指标，它是回头调闸门参数的唯一依据；退避到上限就转降级而不是继续重试。

  Key points

  • Treat throttling as a signal: back off and also penalize the bucket so the next window issues fewer tokens.
  • Requeue with a not-before timestamp instead of sleeping inside the worker slot.
  • Add jitter, or everything throttled together wakes together and collides again.
  • Separate retryable from non-retryable: auth, balance, bad parameters and content rejections get zero retries.
  • Emit a throttle counter as a metric, and switch to degradation once backoff hits its ceiling.