Interview Bank

How to reason about it · think before answering

1. The hinge is the second half. They are checking whether you separate 'await a batch' from 'cap how many run at once' — similar API names, unrelated semantics.
2. Name the wrong answer first: mapping 500 items to promises and awaiting them together runs at concurrency 500. Creating the promise already fired the request; awaiting only collects results. gather and CompletableFuture.allOf are the same trap in other accents.
3. Then give the two correct shapes: a fixed set of workers pulling from a shared cursor (the JS idiom, where a worker is the slot), or a semaphore gating task start (asyncio.Semaphore, java.util.concurrent.Semaphore). Swift needs a manual window over a TaskGroup — fill limit slots, then add one task per result received.
4. The real failure mode is slot leakage: release must happen in a finally, or the error must be collapsed into a result value inside the task. Code that misses this looks perfect on the happy path and only degrades once the downstream starts failing, which makes it one of the hardest bugs to trace.
5. Tie it to agents: batch embedding, parallel tool calls, fan-out subtasks. The benefit is not only sparing the downstream — peak memory now scales with the concurrency limit instead of the task count.
6. Expect the follow-up: what if tasks retry? Retries must happen inside the slot, otherwise a retry storm bypasses the limiter entirely. One level deeper: add jitter so failed tasks do not all come back at the same instant.

分析过程 · 先想清楚再作答

1. 题眼在后半句。面试官在确认你分不分得清「等待一批任务」和「限制同时运行的任务数」——这两件事在 API 名字上很像，在语义上毫无关系。
2. 先说破错误答案为什么错：把 500 个任务全部映射成 Promise 再一起 await，这段代码的并发度是 500。Promise 一被创建，它内部的请求就已经发出去了，await 只是在等结果；gather 和 CompletableFuture.allOf 是同一个坑的另外两种口音。
3. 再给正确形状的两条路：固定数量的工人从同一个游标取任务（JS 的惯用法，槽位就是工人本身），或者用信号量挡在任务启动之前（Python 的 asyncio.Semaphore、Java 的 Semaphore）。Swift 要用 TaskGroup 自己开滑动窗口，先塞满 limit 个、每收一个结果补一个。
4. 本题真正的失分点是槽位泄漏：acquire 之后必须在 finally 里 release，或者把错误在任务内部收敛成结果值。忘了这一步的代码在 happy path 上完全正常，只有下游开始报错时才会一点点变慢直到彻底卡死——这是最难查的那类 bug，因为症状出现在故障之后而不是之中。
5. 落到 Agent 场景说收益：批量 embedding、并行工具调用、多路子任务都靠它。收益不只是「不打爆下游」，还有同时驻留的内存与并发度同阶而不是与任务数同阶。
6. 可以预期的追问：如果任务本身还要重试呢？答案是重试要在槽位内部完成（占着槽位退避重试），否则重试风暴会绕过限流；再追一层就是给重试加抖动，避免所有失败任务在同一时刻一起回来。

How to reason about it · think before answering

1. The hinge is the word shared. Naming a token bucket only answers half of it; the interviewer wants your bucketing dimension and what else sits around the bucket.
2. Dimension first: bucket per vendor and per API family, never one global bucket. Image and speech quotas at the same vendor are separate pools, and merging them lets the tight one throttle the loose one.
3. Then the parts: one bucket is not enough. A token bucket caps rate (calls per minute, a number the vendor sets); a semaphore caps concurrency (how many are in flight, a number you set to protect memory and spend). Rate alone lets a fast endpoint fire hundreds per minute; concurrency alone lets twenty downloads pile up.
4. The engineering detail that decides everything: admission must be non-blocking. If a job is dispatched first and then waits for a token inside the worker, low-priority work pins every worker slot and priority silently stops working. Ask the gate before dispatch, and skip to the next candidate when it says no.
5. Finally the numbers: concurrency should not track CPU cores, since this pipeline is almost all network wait. Derive it from vendor quota and from how much work one failure forces you to redo.
6. Expected follow-up: where does bucket state live. In-memory is fine for one process; across processes it belongs in Redis behind an atomic token-take script, or each process limits itself and the sum still blows the quota.

分析过程 · 先想清楚再作答

1. 这题的题眼是「共用」两个字。只回答一个令牌桶算答了一半，面试官想听的是你按什么维度分桶、以及桶之外还需要什么。
2. 先给维度：限流要按「厂商 + 接口类别」分桶，不能全局一个桶。同一家的图像和语音是两个独立的配额池，混在一起会让紧的那个把松的那个也拖住。
3. 再给部件：一个桶不够，要两个。令牌桶管速率（一分钟发几次，数字是厂商定的），信号量管并发（同一时刻挂着几个，数字是你自己定的用来保护内存和钱包）。只有速率控制的话，快速返回的接口一分钟能发几百次；只有并发控制的话，二十个请求同时在飞会把内存挂满。
4. 然后是关键的工程细节：拿许可的动作必须是非阻塞的。如果任务先被派出去、再在执行流里等令牌，工作槽会被一批低优先任务占死，优先级就静默失效了。正确结构是调度器派活之前先问闸门要许可，拿不到就跳过它去看下一个候选。
5. 最后落到取值：并发上限不该按 CPU 核数定，这条线几乎没有本地计算，全在等网络；它该按「一次失败要重跑多少东西」和厂商配额来定。
6. 可预期的追问是「桶的状态放哪」。单进程放内存就够；多进程要放 Redis，用一个原子脚本取令牌，否则每个进程各限各的，加起来照样超。