Interview Bank

Build an AI Short-Drama Production Pipeline With Agents in 14 Days

D9 Concurrency and Quotas: Starting Multiple Episodes at Once Without Blowing Through Any Provider's Limits

• Beyond backing off and retrying, what else should happen when you get rate limited?收到限流响应之后，除了退避重试还该做什么？

  Common in ChinaCommon overseasDeep dive#rate-limiting#error-handling#retry

  How to reason about it · think before answering

  1. This question separates people who have actually been throttled in production. Exponential backoff with jitter is only the first half of the answer.
  2. Frame it correctly: throttling is a signal, not an error. It says your current send rate exceeds what the vendor will accept right now, so it deserves a feedback action, not just a retry.
  3. Action one is to slow down on purpose: penalize the bucket so the next window or two issues half the tokens. Without that, you finish the backoff and hit the same wall at the same speed.
  4. Action two is to not hold an execution slot while waiting. Requeue the job with a not-before timestamp and hand the slot back immediately.
  5. Action three is classification. Throttling and server errors are retryable; auth failure, insufficient balance, invalid parameters and content-policy rejections are not, and retrying them just repeats one mistake five times while consuming quota. At MiniMax, 1002 is rate limiting and 1039 is the token-per-minute variant, while 1004 is auth, 1008 is balance, 2013 is bad parameters and 1026 or 1027 are content rejections.
  6. Action four is to record throttle counts as a metric. That number is the only evidence you have when you later retune the gate.
  7. Expected follow-up: how to cap the backoff. Cap it at what the business can wait for, then degrade instead of retrying: smaller resolution, shorter duration, or push the job into the next batch.

  分析过程 · 先想清楚再作答

  1. 这题在考你有没有真在生产里被限流打过。只答「指数退避加抖动」是标准答案的前半段，面试官等的是后半段。
  2. 先把限流摆正位置：它不是错误，是信号。它告诉你此刻的发送速率超过了厂商愿意接受的速率。既然是信号，就该有反馈动作，而不只是重试。
  3. 第一个动作是主动降速：把令牌桶罚一档，接下来一两个窗口只发一半令牌。不降速的话，退避结束后你会用同样的速度再撞一次，重试次数越多越糟。
  4. 第二个动作是别在退避里占着执行流。正确做法是把任务重新入队并记一个「不早于」时间戳，槽位立刻还回去给别的任务。
  5. 第三个动作是分类：限流和服务端错误可以重试，鉴权失败、余额不足、参数错误、内容审核不通过一次都不该重试——重试只会让你在一分钟里把同一个错误犯五遍，还白占配额。MiniMax 这边 1002 是限流、1039 是 TPM 维度的限流，1004 鉴权、1008 余额、2013 参数、1026 和 1027 是内容审核。
  6. 第四个动作是把限流次数记进指标。撞得多说明闸门配小了或者配大了，这个数字是你回头调参数的唯一依据。
  7. 可预期的追问是「退避上限怎么定」。定在业务能等的时间上，超过就转降级：换更小的分辨率、更短的时长，或者干脆排到下一批。

  Key points

  • Treat throttling as a signal: back off and also penalize the bucket so the next window issues fewer tokens.
  • Requeue with a not-before timestamp instead of sleeping inside the worker slot.
  • Add jitter, or everything throttled together wakes together and collides again.
  • Separate retryable from non-retryable: auth, balance, bad parameters and content rejections get zero retries.
  • Emit a throttle counter as a metric, and switch to degradation once backoff hits its ceiling.

  答题要点

  • 把限流当信号：退避的同时给令牌桶降档，接下来的窗口只发一半令牌。
  • 退避期间把任务重新入队并记一个不早于时间戳，工作槽立刻还回去。
  • 退避要带抖动，否则同时被限的任务会同时醒来再撞一次。
  • 严格区分可重试与不可重试：鉴权、余额、参数、内容审核一次都不重试。
  • 把限流次数记成指标，它是回头调闸门参数的唯一依据；退避到上限就转降级而不是继续重试。