Interview Bank

From Frontend Engineer to Agent Engineer in 30 Days

D29 Shoring Up Weak Points + a Coding Warm-Up: Rate Limiter, LRU, Concurrency Control, Streaming JSON Parsing

• How would you build a scheduler that caps in-flight async tasks, and why is Promise.all or asyncio.gather not enough?怎么实现一个限制并发数的调度器？为什么不能直接用 Promise.all 或者 asyncio.gather？

  Common in ChinaCommon overseasIntermediate#concurrency#async

  How to reason about it · think before answering

  1. The hinge is the second half. They are checking whether you separate 'await a batch' from 'cap how many run at once' — similar API names, unrelated semantics.
  2. Name the wrong answer first: mapping 500 items to promises and awaiting them together runs at concurrency 500. Creating the promise already fired the request; awaiting only collects results. gather and CompletableFuture.allOf are the same trap in other accents.
  3. Then give the two correct shapes: a fixed set of workers pulling from a shared cursor (the JS idiom, where a worker is the slot), or a semaphore gating task start (asyncio.Semaphore, java.util.concurrent.Semaphore). Swift needs a manual window over a TaskGroup — fill limit slots, then add one task per result received.
  4. The real failure mode is slot leakage: release must happen in a finally, or the error must be collapsed into a result value inside the task. Code that misses this looks perfect on the happy path and only degrades once the downstream starts failing, which makes it one of the hardest bugs to trace.
  5. Tie it to agents: batch embedding, parallel tool calls, fan-out subtasks. The benefit is not only sparing the downstream — peak memory now scales with the concurrency limit instead of the task count.
  6. Expect the follow-up: what if tasks retry? Retries must happen inside the slot, otherwise a retry storm bypasses the limiter entirely. One level deeper: add jitter so failed tasks do not all come back at the same instant.

  分析过程 · 先想清楚再作答

  1. 题眼在后半句。面试官在确认你分不分得清「等待一批任务」和「限制同时运行的任务数」——这两件事在 API 名字上很像，在语义上毫无关系。
  2. 先说破错误答案为什么错：把 500 个任务全部映射成 Promise 再一起 await，这段代码的并发度是 500。Promise 一被创建，它内部的请求就已经发出去了，await 只是在等结果；gather 和 CompletableFuture.allOf 是同一个坑的另外两种口音。
  3. 再给正确形状的两条路：固定数量的工人从同一个游标取任务（JS 的惯用法，槽位就是工人本身），或者用信号量挡在任务启动之前（Python 的 asyncio.Semaphore、Java 的 Semaphore）。Swift 要用 TaskGroup 自己开滑动窗口，先塞满 limit 个、每收一个结果补一个。
  4. 本题真正的失分点是槽位泄漏：acquire 之后必须在 finally 里 release，或者把错误在任务内部收敛成结果值。忘了这一步的代码在 happy path 上完全正常，只有下游开始报错时才会一点点变慢直到彻底卡死——这是最难查的那类 bug，因为症状出现在故障之后而不是之中。
  5. 落到 Agent 场景说收益：批量 embedding、并行工具调用、多路子任务都靠它。收益不只是「不打爆下游」，还有同时驻留的内存与并发度同阶而不是与任务数同阶。
  6. 可以预期的追问：如果任务本身还要重试呢？答案是重试要在槽位内部完成（占着槽位退避重试），否则重试风暴会绕过限流；再追一层就是给重试加抖动，避免所有失败任务在同一时刻一起回来。

  Key points

  • Promise.all and asyncio.gather only wait; the work started when each promise was created, so concurrency equals the task count
  • Two correct shapes: a fixed worker set pulling from a shared cursor, or a semaphore gating task start
  • The slot must be returned on every exit path — finally in Java, async with in Python, error-to-value inside a Swift task, try/catch inside the JS loop
  • A leaked slot shows up as gradual slowdown to a full stall once the downstream starts erroring, and is invisible on the happy path
  • The payoff is peak memory scaling with the concurrency limit rather than the task count; retries must stay inside the slot and carry jitter

  答题要点

  • Promise.all 与 asyncio.gather 只负责等待，任务在被创建的那一刻就已经启动了，并发度等于任务总数
  • 两种正确形状：固定数量的工人从共享游标取任务，或者用信号量挡在任务启动之前
  • 槽位必须在任何退出路径上归还：Java 写在 finally 里，Python 用 async with，Swift 把错误收敛成结果值，JS 在循环里 try 与 catch
  • 槽位泄漏的症状是「下游一开始报错就越来越慢直到卡死」，happy path 完全看不出来
  • 收益是同时驻留的内存与并发度同阶，而不是与任务总数同阶；重试要占着槽位做，并加抖动