Interview Bank

From Frontend Engineer to Agent Engineer in 30 Days

D14 Deployment and Operations: Multi-Worker Compose, Heartbeats, Health Checks, Graceful Shutdown, Dev/Prod Isolation; Week Two Retrospective

• System design: design an IM agent platform where users chat with an AI assistant inside a messaging app. The assistant calls tools, remembers long-term preferences, and proactively pushes scheduled messages. Target 100k daily active users.系统设计：请设计一个 IM Agent 平台——用户在即时通讯软件里和一个 AI 助手对话，助手能调用工具、记住长期偏好、还能定时主动推送。要求支撑十万日活。

  Common in ChinaCommon overseasDeep dive#system-design#distributed-systems#cost#operations

  How to reason about it · think before answering

  1. Do not start drawing. The most common way to fail a design question is to hear the prompt and immediately sketch boxes, only for the interviewer to realise twenty minutes later that you solved a different problem. Spend three to five minutes on four questions: traffic shape (how many concurrent sessions does 100k DAU imply, and what is the peak-to-trough ratio), latency (how fast must first byte be, is streaming required), the nature of the tools (read-only lookups, or writes with side effects), and the compliance boundary on proactive pushes (may you push at night, what is the daily cap). All four change the architecture materially, so asking them is itself worth points.
  2. Then state the trunk in one sentence: stateless ingress, a message bus for decoupling, stateful workers sharded by user, all state in the database. Walk the data flow: the messaging platform's webhook hits ingress, which does only auth, rate limiting, persistence and publish, and returns 202 immediately; the execution side pulls work, runs the agent loop, and streams output fragments back; proactive pushes come from a central scheduler publishing onto the same bus. The load-bearing argument is that ingress latency is bounded while execution latency is not, so putting them in one process means one slow model call occupies a connection that should have returned in milliseconds — say this out loud, it is the premise of the whole answer.
  3. Then justify each module. Storage: sessions, runs and messages, with runs existing separately because only it can answer whether this attempt actually finished; idempotency comes from a unique constraint on runs, not from check-then-insert. Bus: Redis Streams consumer groups for fan-out, at-least-once semantics, with exactly-once manufactured by consumer-side idempotency, and messages that fail three times moved to a dead-letter stream. Ordering: the consumer group's unit of assignment is one message while the business requires serialisation per user, so hash userId into a fixed set of shards and let exactly one worker hold each shard's lease. Memory: embeddings in pgvector, retrieval wrapped as a tool the model chooses to call, with no identity parameter — identity only ever comes from the session.
  4. Treat proactive push as its own section, because it is what separates this from an ordinary chat service. The central scheduler publishes one message on a time match and the execution side is unchanged; the idempotency key is anchored to the scheduled minute, so replaying after a scheduler restart cannot double-send. For compliance you need timezone, quiet hours and a daily cap — and all three must be evaluated before publishing rather than at send time, or you have already paid for the model call before discovering you should not have pushed.
  5. Then volunteer capacity and cost numbers, which is what separates senior candidates. 100k DAU at ten turns each is a million model calls; at roughly a thousand tokens in and out, with input at $0.15 and output at $0.60 per million tokens, that is about $750 a day. That number immediately implies three requirements: meter token usage per call and convert to dollars (otherwise you cannot tell which user or feature is burning money), build tiered degradation (push over-budget users to a cheaper model rather than refusing them), and recognise that context length is the dominant cost lever (so compress history and cap retrieved items).
  6. Land on operability, which is this week's payoff: multiple replicas, heartbeats to surface zombies, readiness probes that only check their own hard dependencies, graceful shutdown so deploys do not cut conversations, and dev/prod isolation via key prefixes. Pair every mechanism with what happens when it fails — leases can split-brain so you need a self-fencing rule and fencing tokens, heartbeats produce false positives so a red dashboard alerts a human rather than auto-draining, shutdown can time out so the wait needs a ceiling. A mechanism without a stated failure mode reads as something you only read about.
  7. Expect, in rough order of frequency: where are the single points (the scheduler is stateless and restartable; Redis and Postgres rely on managed primary/replica); how do you roll out safely (old and new workers coexist and a version field in the message selects the prompt set); what if the user sends another message mid-reply (merge a change of mind within thirty seconds into the same execution rather than running two concurrently); and how would you halve the cost (cache frequent answers, compress history, route simple intents to a smaller model).

  分析过程 · 先想清楚再作答

  1. 先别画图。系统设计题最常见的死法是听完就开始画框，二十分钟后面试官发现你解的是另一道题。花三到五分钟问清四件事：一是流量形状（十万日活对应多少并发会话、峰谷比多少），二是延迟要求（首字节要多快，是否必须流式），三是工具的性质（只读查询还是有写操作和副作用），四是主动推送的合规边界（能不能在深夜推、每天上限几条）。这四个答案会实质改变架构，问它们本身就是分数。
  2. 然后给主干，一句话先定形状：**接入层无状态、消息总线解耦、Worker 有状态且按用户分片、状态全在数据库**。接着按数据流走一遍：IM 平台的 webhook 打到接入层，接入层只做鉴权、限流、落库、投递四件事，立刻返回 202；执行侧从总线取活、跑 Agent 循环、把输出片段回传；主动推送由一个中心调度器按时间投递进同一条总线。**关键论点是接入层耗时确定、执行层耗时不确定，把它们放在一个进程里意味着一次慢的模型调用会占住一个本该毫秒级返回的连接**——这是整道题的立论基础，要主动说出来。
  3. 再逐个模块给出选择和理由。存储：sessions / runs / messages 三张表，runs 单独存在是因为只有它能回答「这次到底跑完没有」，幂等靠 runs 上的唯一约束而不是先查后插。总线：Redis Streams 的消费组做分摊，语义是至少一次，恰好一次靠消费端幂等做出来；反复失败的消息投递三次后进死信流。顺序：消费组的分配单位是一条消息而业务要求的串行单位是一个用户，所以按 userId 哈希到固定数量分片，每个分片同一时刻只有一个 Worker 持有租约。记忆：pgvector 存 embedding，检索包成一个工具交给模型自己决定要不要查，且不给它身份参数——身份只能来自会话。
  4. 主动推送这一块要单独讲透，因为它是这道题区别于普通聊天服务的地方。中心调度器命中时间点后只投一条消息，执行侧照旧；幂等键锚在「计划触发的那一分钟」，所以调度器崩溃重启后回看重放不会重复推送。合规上要有时区、静默时段、每日上限三道闸，而且这三道闸必须在投递前判断而不是在推送时判断——否则你已经花了模型调用的钱才发现不该推。
  5. 然后主动给出容量和成本的数字感，这是高级候选人的分水岭。十万日活、人均十轮对话是一百万次模型调用；按输入输出各一千 token、每百万 token 输入 0.15 美元输出 0.60 美元估算，一天大约七百五十美元。这个数字立刻推出三件事必须做：token 用量要按调用记账并换算成美元（否则你无法定位是哪个用户或哪个功能在烧钱）、要有分层降级（超预算的用户切便宜模型而不是直接拒绝）、以及上下文长度是主要成本杠杆（所以要压缩历史、控制检索条数）。
  6. 最后收在可运维性上，也就是这一周的落点：多副本部署、心跳发现假死、就绪探针只查自己必需的依赖、优雅停机让发版不掐断对话、dev 与 prod 用键名前缀隔离。**每个机制都要配一句「它失效时会怎样」**——租约会脑裂所以要有自杀规则和护栏令牌、心跳会误判所以面板转红只告警不自动摘流量、停机会超时所以等待要有上限。说不出失效模式的机制，面试官会认为你只是读过。
  7. 可以预期的追问，按出现频率排：单点在哪（调度器无状态可重启，Redis 和 Postgres 靠托管服务的主备）；怎么灰度（新旧 Worker 同时在线，靠消息里的版本字段决定走哪套提示词）；用户在助手回复中途又发一句怎么办（三十秒内的改口合并进同一次执行，而不是并发开两个）；成本再降一半怎么做（缓存高频问答、压缩历史、把简单意图路由到小模型）。

  Key points

  • Spend three to five minutes clarifying four things: traffic shape, latency targets, whether tools have side effects, and the compliance boundary on proactive pushes
  • State the trunk in one sentence: stateless ingress, bus for decoupling, stateful workers sharded by user, all state in the database — premised on bounded ingress latency versus unbounded execution latency
  • Storage is sessions/runs/messages with idempotency from a unique constraint on runs; the bus is Redis Streams consumer groups, at-least-once plus consumer idempotency, dead-lettering after three failures
  • Ordering comes from hashing userId into shards plus leases: the consumer group assigns per message while the business serialises per user
  • Memory is pgvector exposed as a tool the model may call, with no identity parameter — identity comes only from the session
  • Proactive push flows through a central scheduler with the idempotency key anchored to the scheduled minute; timezone, quiet hours and daily caps are enforced before publishing
  • Bring numbers: 100k DAU at ten turns is ~1M calls and ~$750/day, which implies metering, tiered degradation, and context length as the main cost lever
  • Land on operability: replicas, heartbeats for zombies, readiness probes scoped to own dependencies, graceful shutdown, dev/prod prefix isolation
  • Pair each mechanism with its failure mode — leases split-brain, heartbeats false-positive, shutdown times out; a mechanism without one reads as book knowledge

  答题要点

  • 先用三到五分钟问清四件事：流量形状、延迟要求、工具是否有副作用、主动推送的合规边界——它们会实质改变架构
  • 主干一句话：接入层无状态、消息总线解耦、Worker 有状态且按用户分片、状态全在数据库；立论是接入层耗时确定而执行层不确定
  • 存储 sessions / runs / messages 三张表，幂等靠 runs 上的唯一约束；总线用 Redis Streams 消费组，至少一次加消费端幂等，三次失败进死信
  • 顺序靠 userId 哈希分片加租约：消费组的分配单位是一条消息，而业务要求的串行单位是一个用户
  • 记忆用 pgvector 并包成工具交给模型自己决定是否检索，不给身份参数——身份只能来自会话
  • 主动推送由中心调度器投递，幂等键锚在计划触发的那一分钟；时区、静默时段、每日上限三道闸必须在投递前判断
  • 给出成本数字感：十万日活人均十轮约一百万次调用、一天约七百五十美元，由此推出计量记账、分层降级、压上下文三件事
  • 收在可运维性：多副本、心跳查假死、就绪探针只查自己的依赖、优雅停机、dev/prod 前缀隔离
  • 每个机制都配一句失效模式：租约会脑裂、心跳会误判、停机会超时——说不出失效模式等于只是读过

D20 Scheduled Jobs and Proactive Outreach: Time Zones, Quiet Hours, Daily Caps, a Notification Provider Abstraction

• How does a system-initiated message differ from a user-triggered one, from a system design point of view?系统主动发给用户的消息，和用户自己触发的消息，在系统设计上有什么不同？

  Common in ChinaCommon overseasBasic#proactive-messaging#system-design#product-engineering

  How to reason about it · think before answering

  1. This looks like a definition question but it is really a filter. Answering both send a message, only the trigger differs stays at the shallowest layer — the interviewer wants to know what extra code the difference forces you to write.
  2. Give three structured differences: who is waiting (a user-triggered reply has someone staring at the screen, a proactive message has nobody waiting); how failure is handled (user-triggered failures must surface as errors, proactive failures should usually be silently deferred or dropped); and what justifies sending (the user asked, versus you having to justify it yourself).
  3. The third is the hinge, so make it explicit: the default answer for a proactive message is do not send. Every one must answer why now, why this user, and why this content is worth interrupting them. Fail any of the three and it should not go out.
  4. Then land the difference in the system: the proactive path needs an admission layer the reactive path does not — compute the user's local time from their timezone, defer if it falls inside quiet hours, drop if the daily cap is used up.
  5. Quantify the cost, which is what separates having read about this from having shipped it: tolerance for proactive messages is very low. After a few irrelevant pushes the user will not argue about the content, they will revoke the notification permission — and once revoked, the genuinely important message cannot reach them either. You are spending a budget that never refills.
  6. Expect: so is a cron job the same thing as a proactive message? No. The scheduler solves firing on time (central scheduling, an idempotency key anchored to the scheduled minute); proactive care solves whether to send at all. One is mechanism, the other is admission, and they belong in separate layers.

  分析过程 · 先想清楚再作答

  1. 这题看着像概念题，其实是筛人题。答「都是发消息，只是触发方不同」就落进了最浅的一层——面试官想听的是这个差别会逼你多写哪些代码。
  2. 先给三条结构化的差别：谁在等（用户触发时他正盯着屏幕，主动消息没有人在等）；失败怎么处理（用户触发的失败必须报错给他看，主动消息的失败多数时候应该安静地推迟或放弃）；凭什么发（用户触发是他开了口，主动消息你得自己说出理由）。
  3. 第三条是题眼，要说透：主动消息的默认答案是不发。每一条都要能回答为什么是现在、为什么是这个用户、为什么这条内容值得打断他，三个问题答不上任何一个就不该发。
  4. 然后给出这个差别在系统里的落点：主动消息这一侧必须多出一层准入判断，本课叫三道闸——按用户时区算本地时间、安静时段命中就推迟、每日上限满了就拦下。用户触发那一侧完全不需要这层。
  5. 代价也要算清楚，这是区分「读过文章」和「做过系统」的地方：用户对主动消息的容忍度极低，连着几条无关紧要的推送之后他不会争论内容对不对，直接关掉通知权限——而权限一关，你连真正重要的那条也送不出去了。你消耗的是一个用完就拿不回来的额度。
  6. 可以预期的追问：那定时任务和主动消息是不是一回事？答不是。定时任务解决的是「能按时触发」（中心调度、幂等键锚在计划触发的那一分钟），主动消息解决的是「该不该发」，前者是机制、后者是准入，两层要分开做。

  Key points

  • Three differences: who is waiting, how failure is handled, and what justifies sending — the third is the crux
  • The default answer for a proactive message is no; each one must justify why now, why this user, why worth interrupting
  • In the system this becomes an admission layer — timezone, quiet hours, daily cap — that the reactive path does not need
  • The cost is a non-renewable budget: annoy the user and they revoke notifications, taking the important messages down with them
  • Scheduling (fire on time) and proactive care (should we send) are two separate layers

  答题要点

  • 三条差别：谁在等、失败怎么处理、凭什么发；第三条是关键
  • 主动消息的默认答案是不发，每条要能回答为什么是现在、为什么是这个用户、为什么值得打断他
  • 落到系统上就是多一层准入判断：时区换算、安静时段、每日上限，用户触发那一侧不需要
  • 代价是一个不可再生的额度：推送惹烦了用户，他关掉权限之后重要消息也送不出去
  • 定时机制（能按时触发）和主动关怀（该不该发）是两层，不要混在一起做

D21 Evaluation and Observability: a Golden Set, LLM-as-Judge, Tracing, a Failure-Rate/Cost Dashboard; Pi vs. LangGraph Summary; Week Three Retrospective

• System design: a multi-agent support platform is live, the team edits prompts several times a week, nobody can say whether quality is improving, and cost is only known as a month-end total. Design its evaluation and observability system.系统设计：一个多 Agent 客服平台已经上线，团队每周改几次提示词，但没人说得清质量是变好还是变差，成本也只有一个月底的总数。请为它设计一套评估与可观测体系。

  Common in ChinaCommon overseasDeep dive#system-design#evaluation#observability#cost

  How to reason about it · think before answering

  1. Do not draw an architecture diagram yet. The trap is that this sounds like build monitoring, so many candidates open with Prometheus and Grafana — that answers infrastructure, not this question. Spend three to five minutes on four things: how often prompts change and how they ship (weekly cadence, canary, rollback); how problems surface today (user complaints, or someone happening to notice); what history exists (how long conversations are retained, whether they can be replayed); and who consumes this (engineers debugging, or an executive watching spend). All four materially change the design, so asking them scores.
  2. Then the trunk, in one sentence: one dataset, two readings. Instrument once, as spans; read across for a single request's call tree (debugging) and stack them for a dashboard (trends and cost). This is the foundation — two data sources will eventually disagree and then nobody trusts either. Many candidates fork here into a monitoring system and an evaluation system, which is the source of every later problem.
  3. Then three layers. Layer one, offline regression: a small stable golden set (15 to 50), covering three things — every route exercised, one item per failure mode (low-confidence fallback, tool budget exhaustion, downstream outage), and the cases behind real past incidents. Each item declares its expected route and a checklist of required facts. The maintenance rule is add, never edit: changing an expectation voids all historical scores. Score with an LLM-as-judge using a different model, and version the rubric, storing that version on every record. This layer runs in CI on every prompt change and emits a number comparable to last time.
  4. Layer two, online observability: every request writes a span tree recording the routing rationale (a model decision, lost forever if not captured), per-node tokens and latency, and degradation and fallback events. The dashboard answers four questions: how much is wrong, where it is slow, what it cost, and which role spent it — that last one is multi-agent specific and the most actionable.
  5. Layer three, online sampled evaluation: fifteen offline cases cannot cover the real traffic distribution, so sample a fraction of live requests (say 1%) through the same judge to get a true quality curve. This layer bridges the other two: offline tells you whether you broke something known, online tells you what real users encountered.
  6. Bring numbers on cost, which is what separates levels. A multi-agent request can produce five to ten model calls, so per-call price is an order of magnitude below the real unit cost and you must price per request. Give the arithmetic: 10k DAU at three sessions each and five calls per session is 150k calls a day; at 2000 input and 500 output tokens, $0.15 and $0.60 per million, that is roughly $90 a day. That number implies two things: per-node attribution shows where to optimise, and evaluation's own cost must be tracked separately, since judge calls are the same order as the system itself and decide whether you evaluate per commit or nightly.
  7. Close on adoption, which many candidates omit: wire evaluation into the release process (block a deploy when pass rate drops below threshold), keep the rubric and golden set in the repository under code review, and pair every mechanism with a failure mode — judges favour same-family models, golden sets get gamed (someone tunes prompts to make it green, and at that moment it is worthless), sampling misses the long tail. A proposal with no stated failure modes reads as book knowledge.
  8. Expect, by frequency: which model judges (one tier above the system under test, and necessarily a different family); where the golden set comes from (start with human-labelled production samples, then append every incident); what happens when this system itself misbehaves (the dashboard refuses to aggregate mixed rubric versions rather than emitting a meaningless average); and how long to build (layer two in a week, layer one in two, layer three in a month since it depends on both).

  分析过程 · 先想清楚再作答

  1. 先别画架构图。这道题的陷阱是它听起来像「搭一套监控」，于是很多人上来就报 Prometheus 加 Grafana——那答的是基础设施，不是这道题。花三到五分钟问清四件事：一是**改提示词的频率和发布方式**（每周几次、有没有灰度、能不能回滚）；二是**现在出问题是怎么发现的**（用户投诉？还是有人偶然看到？）；三是**有没有历史数据**（线上对话存了多久、能不能回放）；四是**谁来看这套东西**（工程师排障，还是老板看成本）。这四个答案会实质改变设计，问它们本身就是分数。
  2. 然后给主干，一句话定形状：**一份数据、两种读法。** 埋点只做一套（span），横着读是一次请求的调用树（排障用），竖着堆是面板（趋势和成本用）。**这条是地基**——两套数据来源迟早对不上，然后没有人相信任何一个。很多候选人在这里就分叉成「监控系统」和「评估系统」两套，那是后面所有麻烦的源头。
  3. 接着按三层展开。**第一层，离线回归**：建一个小而稳的 golden set（15 到 50 条），三层覆盖——每条路由都有人走、每种失败模式各一条（置信度不足落兜底、工具预算耗尽降级、下游挂掉）、以及历史上真出过事故的那几条。每条写清期望路由和必备信息清单。维护规矩是**只增不改**：改一条期望，历史分数全部作废。用 LLM-as-judge 对照清单打分，**judge 换一个模型、rubric 版本化并随每条记录存下来**。这一层挂在 CI 上，每次改提示词跑一遍，产出一个能和上次比的数字。
  4. **第二层，在线观测**：每次请求落一棵 span 树，必须记路由理由（模型做的决策，当时不记就永远丢了）、每个节点的 token 与耗时、以及降级和兜底事件。面板回答四个问题：错了多少、慢在哪、花了多少、**钱花在哪个角色身上**。最后一个是多 Agent 特有的，也最有用。
  5. **第三层，在线采样评估**：离线的 15 条覆盖不了真实流量分布，所以按比例采样线上请求（比如 1%）跑同一套 judge，得到一条真实质量曲线。**这一层是前两层的桥**：离线告诉你有没有改坏已知的东西，在线告诉你真实用户遇到了什么。
  6. 成本这块要给数字感，这是区分层级的地方。**多 Agent 一次用户请求可能产生 5 到 10 次模型调用**，所以「每次调用多少钱」比真实单价小一个数量级，**必须按请求算钱**。给个算式：日活一万、人均三次会话、每次 5 次调用就是 15 万次调用；按输入 2000 输出 500 token、$0.15/$0.60 每百万算，一天约 90 美元。这个数立刻推出两件事：按节点分摊能定位省钱的地方，以及**评估本身的成本要单独记**——judge 调用和被评估系统一个量级，它决定你每次提交都跑还是每天跑一次。
  7. 最后收在「怎么让它真的被用起来」，这是很多人漏的一层：把评估结果接进发布流程（通过率跌破阈值就挡住发布）、把 rubric 和 golden set 放进代码仓库走 code review、以及**给每个机制配一句失效模式**——judge 会偏向同源模型、golden set 会被针对性优化（有人为了让它绿而调提示词，那一刻它就失去了意义）、采样会漏掉长尾。说不出失效模式的方案，面试官会认为你只是读过。
  8. 可以预期的追问，按频率排：judge 用什么模型（比被评估的强一档，且必须异源）；golden set 从哪来（先从线上捞一批人工标注，再逐次把事故补进去）；这套东西自己出问题怎么办（面板发现 rubric 混版直接拒绝聚合，而不是给一个没含义的平均分）；多久能上线（第二层一周、第一层两周、第三层一个月，因为它依赖前两层）。

  Key points

  • Spend three to five minutes clarifying four things: prompt change cadence and release process, how problems surface today, what replayable history exists, and who the audience is
  • The trunk is one dataset, two readings: instrument once as spans, read across for a call tree and stack for a dashboard; two sources will disagree
  • Layer one, offline regression: a small stable golden set covering every route, every failure mode and past incidents, add-never-edit, wired into CI
  • Layer two, online observability: span trees recording routing rationale, per-node tokens and latency, degradation events; the dashboard answers wrong/slow/cost/which-role
  • Layer three, sampled online evaluation through the same judge, covering the real distribution the offline set cannot
  • Price per request, not per call: five to ten calls per request, with arithmetic showing ~$90/day at 10k DAU; track evaluation's own cost separately
  • Close on adoption: block releases when pass rate drops, keep rubric and golden set in the repo under review
  • Pair every mechanism with a failure mode: judge self-preference, golden set gaming, sampling missing the tail — omitting these reads as book knowledge

  答题要点

  • 先用三到五分钟问清四件事：改提示词的频率与发布方式、现在问题怎么被发现、有无历史数据可回放、这套东西给谁看
  • 主干是「一份数据、两种读法」：埋点只做一套 span，横着读是调用树、竖着堆是面板；两套数据源迟早对不上
  • 第一层离线回归：小而稳的 golden set，三层覆盖（每条路由、每种失败模式、历史事故），只增不改，挂 CI
  • 第二层在线观测：span 树记路由理由、每节点 token 与耗时、降级兜底事件；面板回答错了多少/慢在哪/花了多少/钱花在哪个角色
  • 第三层在线采样评估：按比例采样线上请求跑同一套 judge，补上离线覆盖不到的真实分布
  • 成本必须按请求算而非按调用：一次请求 5 到 10 次调用，给出日活一万约 90 美元一天的算式；评估自身成本单独记
  • 收在落地：通过率跌破阈值挡发布、rubric 与 golden set 进仓库走 review
  • 每个机制配失效模式：judge 偏向同源、golden set 会被针对性优化、采样漏长尾——说不出失效模式等于只是读过

D26 System Design Deep Dive: Agent Platforms / Customer-Support Agents / Multi-Tenancy / Cost Control

• You get 35 to 40 minutes for a system design round. How do you budget that time, and why is drawing the architecture not step one?系统设计环节只有 35 到 40 分钟，你会怎么分配时间？为什么第一步不是画架构图？

  Common in ChinaCommon overseasBasic#system-design#interview-process

  How to reason about it · think before answering

  1. This question tests pacing, not knowledge. Interviewers ask it because the previous candidate spent 25 minutes on the architecture diagram and left five each for deep dives and trade-offs — which is exactly where the rubric puts most of the weight.
  2. Give the structure with explicit time boxes: 5 minutes clarifying requirements, 3 minutes on capacity and cost estimation, 8 minutes sketching the architecture, 15 minutes going deep on two or three areas, 5 minutes on trade-offs. Naming actual minute counts is itself worth points, because it shows you have rehearsed against a clock.
  3. Then answer the 'why not draw first' half head on: a one-line prompt leaves five things unknown — daily actives, latency budget, cost budget, multi-tenancy, and failure tolerance — and every one of them changes the architecture materially. Drawing first means at best you guessed right, at worst the interviewer realises twenty minutes in that you solved a different problem. An analogy lands it: the client said 'we need an office building' and you unrolled construction drawings before hearing whether the budget is twenty million or two hundred million.
  4. Add the situation that comes up almost every time: you start asking and the interviewer says 'just assume something'. That is not permission to skip clarification, it is an invitation to state a number and its justification. The right reply is 'then I will assume 10k daily actives at five turns each, and I will flag in the final step what changes at 100k'. You keep the pacing and turn the assumption into a traceable premise.
  5. Close by explaining how step four is prepared: those 15 minutes cannot be improvised. Have three deep-dive packages ready — state and ordering, cost and rate limiting, failure and retry — so any pick is covered. Saying you prepared three directions signals rehearsal better than winging one.
  6. Expect the follow-up: what if you run out of time? Cut step three, never step five. An unfinished sketch can be closed with 'the rest follows the standard pattern, happy to come back to it', but dropping the trade-off section makes you indistinguishable from someone who memorised an architecture.

  分析过程 · 先想清楚再作答

  1. 这题考的不是知识，是节奏感。面试官问它，通常是因为上一位候选人在架构图上讲了 25 分钟，深入和权衡各剩五分钟——而评分表上分数最重的恰恰是后两步。
  2. 先给结构，五步加时间盒：需求澄清 5 分钟、容量与成本估算 3 分钟、架构草图 8 分钟、深入 2 到 3 个点 15 分钟、权衡与取舍 5 分钟。给得出具体分钟数本身就是分数，因为它说明你掐过表。
  3. 然后正面回答「为什么不先画图」：一句话的题干里，日活、延迟预算、成本预算、是否多租户、失败可容忍度这五件事全是未知的，而它们每一个都会实质改变架构。不问就画，最好的结果是运气好蒙对，最坏的结果是二十分钟后面试官发现你解的是另一道题。用一个类比说清：甲方只说「我要一栋办公楼」，你就展开施工图，而他连预算是两千万还是两个亿都没讲。
  4. 补一条几乎每次都会遇到的现场情况：你开始问，面试官说「你先自己假设一个」。这不是让你别问了，是让你自己给一个数并说出依据。正确接法是「那我按日活 1 万、人均 5 轮算，如果实际是十万级我会在最后一步说明哪里要改」——既守住了节奏，又把假设变成了可追溯的前提。
  5. 最后主动交代第四步的准备方式：深入的 15 分钟不能临场想，要提前备好三个「深入包」（状态与保序、成本与限流、失败与重试），面试官挑哪个都有货。说得出「我提前准备了三个方向」，比现场硬讲一个更能体现你练过。
  6. 可以预期的追问：如果时间不够怎么办？答案是砍第三步而不是砍第五步——草图讲不完可以说「其余按常规做，需要的话我们回头补」，但权衡那 5 分钟一旦砍掉，你就和一个只会背架构的人没有区别。

  Key points

  • Five steps with time boxes: clarify 5, estimate 3, sketch 8, deep dive 15, trade-offs 5
  • Do not sketch first because DAU, latency budget, cost budget, multi-tenancy and failure tolerance all change the architecture
  • When told to 'just assume something', state a number with its justification instead of skipping clarification
  • Fill the 15-minute deep dive from three pre-prepared packages: state and ordering, cost and rate limiting, failure and retry
  • If time runs short, cut the sketch, never the trade-offs — almost nobody does that section, so doing it stands out

  答题要点

  • 五步加时间盒：澄清 5 分钟、估算 3 分钟、草图 8 分钟、深入 15 分钟、权衡 5 分钟
  • 不先画图，是因为日活、延迟预算、成本预算、是否多租户、失败可容忍度这五件事都会实质改变架构
  • 面试官说「你先假设一个」时，要自己给数并说出依据，而不是跳过澄清
  • 深入的 15 分钟要靠提前备好的三个「深入包」：状态与保序、成本与限流、失败与重试
  • 时间不够时砍草图不砍权衡——权衡那 5 分钟几乎没人做，做了就是加分
• System design: design an e-commerce customer support agent. It looks up orders and shipments, drafts refunds by policy, answers product and policy questions, and escalates to a human when it cannot resolve the issue.系统设计：请设计一个电商客服 Agent。它要能查订单和物流、按规则拟退款方案、回答商品与政策问题，并在搞不定时转人工。

  Common in ChinaCommon overseasDeep dive#system-design#customer-support#escalation

  How to reason about it · think before answering

  1. Start by separating this from 'design an agent platform', or you will answer an infrastructure question. The platform question is about running execution reliably; this one is about not trapping users inside a bot. The rubric lives in the business exits, not the message bus. So pin the thesis in your first sentence: every conversation must end in exactly one of three exits — self-served, handed to a human, or filed as a ticket.
  2. Clarify for 5 minutes, asking four things: daily actives and concurrent sessions (does execution need to be split out), whether human agents work nights (does exit three exist), whether the agent executes refunds or only drafts them (do you need an approval tier), and how large the knowledge base is and how often it changes (is retrieval the centre of this problem). The third question matters most: it decides whether this system has irreversible side effects.
  3. Estimate for 3 minutes, out loud: 2000 input plus 500 output per turn, input is 2000 over a million times $0.15 which is $0.0003, output is 500 over a million times $0.60 which is also $0.0003, so about $0.0006 per turn. 10k daily actives at five turns is 50k turns, roughly $30 a day and $900 a month in model spend, machines excluded. For concurrency, a peak factor of 3 and 6 seconds per turn gives about 11 in-flight executions at peak, which is 3 worker replicas at 4 concurrent each. State the arithmetic before the result — the interviewer's next line is always 'where did that number come from'.
  4. Sketch for 8 minutes, four blocks: ingress does auth, rate limiting, persistence and publish, then returns immediately; execution pulls from the bus, runs the agent loop, and streams sequenced fragments back; storage is sessions, runs and messages plus a chunk table for the knowledge base; observability is tracing plus a cost ledger. Then mark on the diagram which node decides between the three exits — that single annotation tells the interviewer you are answering the support question rather than the generic platform one.
  5. Go deep for 15 minutes, starting with escalation because that is the crux. The criteria must be quantified, any one of four triggering a handoff: two consecutive unresolved turns, an explicit user request, an amount above the auto-execution ceiling (50 CNY in our setup), or a sentiment keyword hit. Then describe the handoff payload: not forty turns of raw transcript, but a structured summary — the user's ask in one line, verified facts, actions already taken, and the failure reason, with a link to the full transcript. Cover the knowledge base in one line (hybrid search, rerank, inline citations) and spend the weight on 'when the citation set comes back empty, take exit two or three rather than letting the model invent an answer' — that is the sentence they will push on. Cover multi-turn in one line too: compress once history passes 70% of budget, cut on a turn boundary, and merge a change of mind within 30 seconds into the same execution.
  6. Trade-offs for 5 minutes, three points: bias the escalation threshold toward escalating, because a false handoff costs one human conversation while trapping a user costs a churned customer and a bad review — different orders of magnitude. Drafting refunds instead of executing them trades one human approval for an entire class of irreversible incidents. And name what breaks the design: once the agent team is large enough to need skill-based routing and queueing, escalation stops being a boolean and becomes its own scheduling system.
  7. Expect, in rough order: does 'I want to file a complaint' count as a sentiment hit (yes, and track that class separately — it is a product signal); should the agent keep listening after handoff (yes, to summarise and prompt the human, but not to speak); how do you stop users being bounced repeatedly (allow one handoff per conversation, then file a ticket); and what happens to old answers when the knowledge base changes (cite chunk ids and versions so you can trace which revision was wrong).

  分析过程 · 先想清楚再作答

  1. 先说这题和「设计一个 Agent 平台」的区别，否则你会把它答成一道基础设施题。平台题考的是怎么把执行跑稳，这题考的是**怎么保证不把用户困在机器人里**——面试官心里的评分点在业务出口上，不在消息总线上。所以主线要一开口就钉死：任何一通会话最后只能落到三条出口之一，自助解决、转人工、留工单。
  2. 第一步澄清 5 分钟，问四件事：日活与并发会话数（决定要不要拆执行层）、人工坐席有没有夜班（决定出口三存不存在）、退款是 Agent 直接执行还是只拟方案（决定要不要人工确认档）、知识库有多大且多久更新一次（决定检索是不是本题的重点）。第三个问题尤其关键，它直接决定这道题是不是带副作用。
  3. 第二步估算 3 分钟，现场算：单轮 2000 输入加 500 输出，输入 2000 除以一百万乘 0.15 等于 0.0003 美元，输出 500 除以一百万乘 0.60 也等于 0.0003 美元，一轮约 0.0006 美元；日活 1 万、人均 5 轮就是 5 万轮，一天约 30 美元、一个月约 900 美元，模型费不含机器。并发按峰谷比 3、单轮 6 秒算，峰值在途约 11 次执行，每个 worker 并发 4 就是 3 个副本。报数字之前先报算式，面试官插的那句一定是「这个数怎么来的」。
  4. 第三步草图 8 分钟，四块：接入层只做鉴权、限流、落库、投递并立刻返回；执行层从消息总线取活跑 Agent 循环、片段带序号回传；存储是会话、执行、消息三张表加一张知识库切块表；可观测是 tracing 加成本台账。在这张图上额外标出三条出口的分叉点在哪一个节点上——这是本题独有的一笔，画上去面试官立刻知道你答的是客服而不是通用平台。
  5. 第四步深入 15 分钟，优先讲转人工这一支，因为它是本题的题眼。判据必须量化，四条任一命中就转：连续 2 轮未解决、用户明确要求、涉及金额超过自动执行上限（本课口径 50 元）、情绪词命中。接着讲交接形状——不是把 40 轮原文丢给客服，而是一段结构化摘要：用户诉求一句、已核实事实几条、Agent 已做过的动作、失败原因，附原始对话链接。知识库那一支一句话带过混合检索加重排加引用，重点落在「引用为空时走出口二或三，而不是让模型编一个答案」，这是最容易被追的一句。多轮那一支同样一句话：历史超七成预算触发压缩且切口对齐到一轮开头，用户中途改口则 30 秒内合并进同一次执行。
  6. 第五步权衡 5 分钟，说三件事：转人工的判据宁可偏松，因为误转的代价是一次人工会话，把用户困住的代价是一个流失客户加一条差评，两者不在一个量级；退款只拟方案不直接执行，是拿一次人工点头换掉一整类不可逆事故；以及什么规模会推翻这个设计——坐席团队大到需要技能路由和排队策略时，转人工就不再是一个布尔判断，而是另一套调度系统。
  7. 可以预期的追问，按频率排：用户说「我要投诉」算不算情绪词命中（算，且这一类要单独统计，它是产品问题的信号）；转人工之后 Agent 还要不要继续在旁边听（要，用来生成小结和给坐席提示，但不允许再发言）；怎么防止用户被反复转来转去（同一通会话只允许转一次，第二次直接留工单）；以及知识库更新后旧答案怎么办（回答里带引用编号和版本，出问题能倒查是哪一版说错的）。

  Key points

  • Thesis: every conversation ends in exactly one of three exits — self-served, escalated to a human, or filed as a ticket
  • Clarify four things: concurrent sessions, whether humans cover nights, whether refunds are executed or only drafted, and knowledge base size and churn
  • Estimate with arithmetic: about $0.0006 per turn, so 10k DAU at five turns is roughly $30/day and $900/month; peak concurrency about 11, meaning 3 worker replicas
  • Quantify escalation: two consecutive unresolved turns, an explicit request, an amount over the auto-execution ceiling, or a sentiment keyword
  • Hand over a structured summary — ask, verified facts, actions taken, failure reason — plus a transcript link, not forty raw turns
  • When retrieval returns no citations, take exit two or three instead of letting the model improvise; answers carry citation ids
  • Reuse compression and 30-second merge for multi-turn; draft refunds rather than executing them, trading one approval for a class of irreversible incidents
  • Trade-off: bias toward escalating, because a false handoff and a trapped user cost different orders of magnitude

  答题要点

  • 主线一句话：任何一通会话只能落到三条出口之一——自助解决、转人工、留工单
  • 澄清必问四件事：并发会话数、人工有没有夜班、退款是执行还是只拟方案、知识库规模与更新频率
  • 估算带算式：单轮约 0.0006 美元，日活 1 万人均 5 轮约 30 美元一天、900 美元一月；峰值并发约 11、3 个 worker 副本
  • 转人工判据必须量化，四条任一命中：连续 2 轮未解决、用户明确要求、金额超自动执行上限、情绪词命中
  • 交接给人工的是结构化摘要（诉求、已核实事实、已做动作、失败原因）加原始对话链接，不是 40 轮原文
  • 知识库检索不到时走出口二或三，绝不让模型自由发挥编答案；回答带引用编号
  • 多轮沿用压缩与 30 秒打断合并，退款只拟方案不直接执行，用一次人工点头换掉一类不可逆事故
  • 权衡：判据宁可偏松，因为误转和困住用户的代价不在一个量级
• For a multi-tenant agent service, how do you design data isolation and billing isolation, and when do you move from a shared table to a dedicated database per tenant?一个多租户的 Agent 服务，数据隔离和计费隔离要怎么设计？什么时候该从共享表升级到独立库？

  Common in ChinaCommon overseasIntermediate#system-design#multi-tenancy#isolation

  How to reason about it · think before answering

  1. The hinge is that 'isolation' is plural. Plenty of candidates answer only data isolation, but the layer that actually breaks in production is resources: one tenant's spike starves everyone else, no rows leak, and users still complain. Open with all three — data, resources, billing — and note that each missing layer maps to its own class of incident.
  2. On data, one sentence separates people who shipped this from people who read about it: 'every query carries tenant_id' versus 'row-level security is the backstop'. The first eventually misses a query, and the one it misses is always the newest, least-tested feature. The correct framing is that RLS is the gate and the application-level where clause is just an optimisation — the same reasoning as idempotency being adjudicated by a database unique constraint. Whatever the data layer can enforce should not depend on everyone remembering.
  3. On resources, give two concrete things: a rate-limit bucket per tenant, and workers sharded by a hash of the tenant id. That is the same sharding mechanism used to preserve per-user ordering, with a different hash input and a different purpose — containing spikes rather than serialising. Noisy neighbours hurt more in agent workloads because a single execution can run thirty seconds, so a thousand queued items from one tenant leaves everyone else waiting.
  4. Billing is the simplest and the most often forgotten: add a tenant column to the token usage ledger and tag every write. Invoicing, quotas and over-budget degradation all hang off it. Bring the cost figures too — roughly $0.0006 per turn at 2000 in and 500 out, about $900 a month at 10k daily actives and five turns each — because quoting per-tenant economics shows you actually ran the numbers.
  5. Then the escalation criteria, the second discriminator. Three tiers: shared table with a tenant column, schema per tenant, database per tenant. The trigger is not tenant count, it is whether a single tenant can starve the rest and whether there is a hard compliance requirement. 'Split the database past a hundred tenants' is guesswork: a hundred small tenants share a table happily, while one regulated enterprise customer may require physical separation on its own. State the cost too — a database per tenant looks clean, but migrations, backups, monitoring and connection pools all multiply by tenant count, so operational cost jumps rather than scaling linearly.
  6. Expect the sharpest follow-up: does the idempotency key change under multi-tenancy? The algorithm does not, but its scope must include the tenant id. Without it, two tenants whose clients independently produce the same string — both using order id order-1024 — collide, and the later request is rejected by the unique constraint as a duplicate. One tenant's write is swallowed by another tenant's history, both logs look perfectly normal, and it is the hardest class of multi-tenant bug to find.

  分析过程 · 先想清楚再作答

  1. 这题的题眼在「隔离」是复数。只答数据隔离的候选人非常多，而多租户翻车最多的其实是资源那一层——一个租户的洪峰打穿别人的处理能力，数据一条都没串，用户照样投诉。所以第一句先把三层摆出来：数据、资源、计费，缺哪一层对应一类事故。
  2. 数据这一层，判断一个人有没有真做过就看一句话：他说「每条查询都带 tenant_id」还是「靠数据库的行级安全兜底」。前者迟早会漏一处，而漏掉的那处通常是最新加、最没被测过的功能。正确说法是行级安全是闸门，应用层那句 where 只是优化——这和幂等的最终裁判必须是数据库唯一约束，是同一种思路：能在数据层强制的，不要指望每个人写代码时都记得。
  3. 资源这一层给两件具体的东西：每个租户一个独立限流桶，以及 worker 按租户标识哈希分片。分片这一招和「按用户哈希保住同一用户顺序」是同一套机制，只是哈希的输入换了，目的从保序变成隔离洪峰。Agent 场景里噪声邻居格外突出，因为单次执行可能跑三十秒，一个租户灌一千条进来，别人就得排队。
  4. 计费这一层最简单也最容易漏：token 用量台账加一列租户标识，写入时打标。账单、配额、超支降级三件事全靠它。顺带说一句成本口径——单轮 2000 输入加 500 输出约 0.0006 美元，日活 1 万人均 5 轮约每月 900 美元，能报出这个量级说明你真的算过每租户成本。
  5. 然后回答升级判据，这是本题的第二个区分点。三档是共享表加租户列、schema 级、库级；**判据不是租户数量，是「有没有单个租户能把别人拖垮」和「有没有合规硬要求」**。答「超过一百个租户就该分库」是典型的凭感觉，因为一百个小租户共享一张表毫无问题，而一个受监管的大客户哪怕只有一个也可能必须物理隔离。代价要一起说：库级隔离看着干净，但迁移脚本、备份、监控、连接池全部乘以租户数，运维成本是陡增不是线性。
  6. 可以预期的追问，也是最见功力的一问：幂等键在多租户下要不要变？答案是键的算法不用变，但**作用域必须带上租户标识**。不带的话两个租户的客户端各自生成了同一个字符串（都用订单号 order-1024），后来那个会被唯一约束当成重复请求挡掉——一个租户的写入被另一个租户的历史请求吞掉，两边日志都完全正常，是多租户里最难查的一类 bug。

  Key points

  • Three parallel layers, each missing one causing its own class of incident: data, resources, billing
  • Data isolation is backstopped by row-level security; the application where clause is only an optimisation
  • Resource isolation is a per-tenant rate-limit bucket plus sharding workers by tenant hash, aimed at noisy neighbours
  • Billing isolation is a tenant column on the usage ledger, powering invoices, quotas and degradation
  • Escalate to schema or database isolation based on starvation risk and compliance mandates, not tenant count
  • Per-tenant databases multiply migrations, backups, monitoring and connection pools — operational cost jumps
  • The idempotency key algorithm stays, but its scope must include the tenant id or identical keys across tenants collide

  答题要点

  • 三层隔离并列，缺一层对应一类事故：数据、资源、计费
  • 数据靠行级安全兜底，应用层的 where 只是优化——能在数据层强制的不要靠人记得
  • 资源是每租户独立限流桶加按租户标识哈希分片，防的是噪声邻居而不是数据串
  • 计费是台账加一列租户标识，账单、配额、超支降级全靠它
  • 升级到 schema 级或库级的判据是「单租户能否拖垮别人」与「有没有合规硬要求」，不是租户数量
  • 库级隔离的代价是迁移、备份、监控、连接池全部乘以租户数，运维成本陡增
  • 幂等键算法不变，但作用域必须带租户标识，否则两个租户的同名键会互相挡掉请求
• An agent system's model spend is out of control. Which levers do you pull, in what order, and roughly how much does each save?一个 Agent 系统的模型成本失控了，你会从哪几个层面着手控制？每一层大概能省多少？

  Common in ChinaCommon overseasIntermediate#system-design#cost#capacity-planning

  How to reason about it · think before answering

  1. The reflex answer is 'switch to a cheaper model', and it is also the easiest one to get killed on: the follow-up is 'how do you know quality did not drop', and without an offline eval set and a comparison run you are exposed. The right opening is 'look at the ledger first' — slice by user, by day and by model to find which dimension is growing. Locate before you act.
  2. Second, put the baseline on the table, because cost talk without a baseline is noise. At 2000 input and 500 output tokens per turn, input is 2000 over a million times $0.15 which is $0.0003, output is 500 over a million times $0.60 which is also $0.0003, so about $0.0006 per turn. 10k daily actives at five turns is 50k turns, roughly $30 a day and $900 a month.
  3. Then give five layers ordered by the cost you pay, not by the savings: caching and prompt caching, tiered model routing, context compression, step and tool budget caps, rate limiting and degradation. The ordering is part of the answer, because it also communicates your rollout sequence.
  4. Attach a number derived from the baseline to each layer. Caching at a 15% hit rate takes $900 to roughly $765. For tiered routing, state the precondition honestly: it is the only layer that can change the order of magnitude, but only if your baseline runs a flagship model — if you already run the cheapest tier there is nothing left to squeeze. Saying that out loud is far more credible than inventing a savings percentage. Compression takes input from 2000 to 1200 tokens, so $0.00048 per turn, about $720 a month, a 20% cut.
  5. Layer four is usually mis-sold as savings; what it actually buys is predictability. With a cap of five tool calls per subtask, per-turn cost finally has a ceiling: five calls each feeding back 800 tokens pushes input to 6000, so $0.0012 per turn, exactly double the baseline — and with no cap there is no ceiling at all. The right phrasing is 'this does not save money, it makes the bill predictable'.
  6. Layer five is rate limiting and degradation, last because it costs the most: $900 across 10k daily actives is about $0.09 per user per month, so a $1 monthly hard cap is invisible to real users and only stops scripted abuse. The nuance is degrade before refusing — this is the only layer users can feel.
  7. Expect the follow-up: which layer first? Say layers one and three, because they only touch your own code, change no product promise, and need no quality re-validation, whereas tiered routing needs an eval set and rate limiting needs product buy-in.

  分析过程 · 先想清楚再作答

  1. 这题最容易脱口而出的答案是「换个便宜模型」，也是最容易被追死的答案——面试官紧跟着就问「你怎么知道换了质量不掉」，答不出离线评估集和对比实验就露馅了。正确的第一句是「先看台账」：按用户、按天、按模型各切一刀，找出是哪一维在涨。先定位再动手，这是工程习惯。
  2. 第二步是把基准摆到桌上，没有基准的成本讨论全是废话。单轮 2000 输入加 500 输出，输入 2000 除以一百万乘 0.15 等于 0.0003 美元，输出 500 除以一百万乘 0.60 也等于 0.0003 美元，一轮约 0.0006 美元；日活 1 万、人均 5 轮就是 5 万轮，一天约 30 美元、一个月约 900 美元。
  3. 然后给五层，排序的依据是**你要付出的代价从小到大**，不是省钱多少：缓存与 prompt cache、模型分级路由、上下文压缩、步数与工具预算上限、限流与降级。这个顺序本身就是答案的一部分，因为它同时说明了你的落地顺序。
  4. 每层配一个从基准推出来的数字。缓存按一成半命中估，900 降到 765 左右。分级路由要诚实说清前提：它是唯一能改数量级的一层，但前提是你的基准用的是旗舰模型；基准已经是最便宜那档时这一层榨不出东西——主动说破这一条，比硬编一个省钱比例可信得多。上下文压缩把输入从 2000 压到 1200，单轮变成 0.00048 美元，一个月 720 美元，降两成。
  5. 第四层最容易被讲成「省钱」，其实它买的是**可预测**：给每个子任务设 5 次工具调用上限之后，单轮成本才有上界——调满 5 次、每次结果回灌 800 token，输入涨到 6000，单轮 0.0012 美元，正好是基准的两倍；没有上限时这个数字没有上界。这一层的正确说法是「我不是靠它省钱，我是靠它让账单可以被预测」。
  6. 第五层是限流与降级，代价最大所以放最后：900 美元摊到 1 万日活是每人每月 0.09 美元，给单用户设 1 美元硬顶，正常用户碰不到，挡的是脚本刷接口那种极端户。要点是超预算先降档再拒绝，而不是直接拒绝——它是五层里唯一用户能感觉到的一层。
  7. 可以预期的追问：这五层里哪一层最先做？答「第一层和第三层」，因为它们只改自己的代码、不动产品承诺、也不需要重新验证质量；而分级路由要配离线评估集，限流要配产品沟通，都不是当天能上的。

  Key points

  • Open with 'look at the ledger', not 'use a cheaper model': slice by user, by day and by model to locate the growth
  • Set a baseline: about $0.0006 per turn, roughly $30/day and $900/month at 10k DAU and five turns
  • Five layers ordered by cost to you: caching and prompt cache, tiered routing, context compression, step and tool budget caps, rate limiting and degradation
  • Tiered routing is the only order-of-magnitude lever, but only if the baseline is a flagship model — say so when it is not
  • Compression from 2000 to 1200 input tokens gives $0.00048 per turn, about $720/month, a 20% cut
  • Tool budget caps buy predictability: with a cap the per-turn ceiling is $0.0012, without one there is no ceiling
  • Rate limiting comes last because users feel it; degrade before refusing

  答题要点

  • 第一句不是「换便宜模型」，是「先看台账」：按用户、按天、按模型各切一刀定位是哪一维在涨
  • 先立基准：单轮约 0.0006 美元，日活 1 万人均 5 轮约每天 30 美元、每月 900 美元
  • 五层按代价从小到大：缓存与 prompt cache、模型分级路由、上下文压缩、步数与工具预算上限、限流与降级
  • 分级路由是唯一能改数量级的一层，但前提是基准用的是旗舰模型；基准已经最便宜时要诚实说没得省
  • 上下文压缩把输入从 2000 压到 1200，单轮 0.00048 美元、每月 720 美元，降两成
  • 工具预算上限买的是可预测：有上限时单轮上界是 0.0012 美元，没上限时没有上界
  • 限流降级放最后，因为它是唯一用户能感觉到的一层；超预算先降档再拒绝

RAG in 14 Days: From Retrieval to Trustworthy Answers

D1 Why Retrieve at All: Hallucination, Knowledge Cutoffs, and the Cost of Long Context; a Minimal Keyword-Only RAG

• Context windows are now in the millions of tokens. Does that make the retrieval step obsolete?上下文窗口已经做到上百万 token 了，检索这一步会被淘汰吗？

  Common in ChinaCommon overseasDeep dive#long-context#cost#system-design

  How to reason about it · think before answering

  1. This is a position question and it is easy to answer as a binary. The signal is whether you separate what fits technically from what is worth paying for on every request.
  2. Concede the valid half first: bigger windows genuinely absorb part of the use case. For an internal tool over a few dozen stable documents with low traffic, stuffing everything in is the right call and building a retrieval stack would be over-engineering.
  3. Then give three reasons it does not absorb the rest. Cost is the first: context is billed per request, so the same corpus is paid for on every one of ten thousand queries, whereas retrieval only pays for the passages it returns. Prompt caching softens this but does not remove it.
  4. Scale is the second: enterprise corpora run to hundreds of thousands of documents and no window holds them. Attribution and access control are the third: pointing an answer at a specific passage, and showing each user only what they are permitted to see, both have to happen before the material reaches the model.
  5. Add the empirical point: as the supplied material grows, models become less reliable at locating the one relevant fact inside it. More context is not automatically better; fewer and more precise passages often win.
  6. Expected follow-up: does retrieval change shape? Yes. Larger windows allow bigger chunks and more of them, which relieves pressure on reranking and compression. Retrieval gets coarser, it does not disappear.

  分析过程 · 先想清楚再作答

  1. 这是一道立场题，容易答成非黑即白。判断你有没有做过的地方在于：会不会区分「技术上能不能塞进去」和「工程上该不该每次都塞」，只谈前者的答案一听就是纸上谈兵。
  2. 先承认对方有道理的部分：窗口变大确实吃掉了检索的一部分场景。几十篇文档、更新不频繁、调用量不大的内部工具，直接全塞是最省事的选择，为它建一套检索系统是过度设计。
  3. 再给三条它吃不掉的理由。第一是成本：材料是按次计费的，同一份材料被问一万次就要付一万次，而检索只付取回的那几段；预填充缓存能缓解但不能消除，缓存也有有效期和命中率。
  4. 第二是规模：企业知识库动辄几十万篇，再大的窗口也塞不下，检索是唯一的入口。第三是归因与权限：答案要指回具体某一段，以及不同的人只能看到自己有权访问的材料——这两件事必须在把材料喂给模型之前完成，窗口再大也不解决。
  5. 还要补一条经验事实：材料变多之后，模型在长上下文里定位关键信息的稳定性会下降，出现「读了但没读到」。所以「全塞」并不总是等于「效果更好」，很多时候少而准反而更好。
  6. 可预期的追问：那检索的形态会不会变？会——窗口变大之后，取回的块可以更大、条数可以更多，重排与压缩的压力变小，检索从「精挑几句」变成「粗筛一批」。趋势是检索的粒度变粗，不是检索消失。

  Key points

  • Separate whether it fits from whether it is worth paying for on every request.
  • Small, stable, low-traffic corpora can legitimately be stuffed whole; building retrieval for them is over-engineering.
  • Three reasons retrieval survives: per-request cost, corpora too large for any window, and attribution plus access control that must happen before the model sees the material.
  • More supplied context reduces the reliability of locating a single fact, so stuffing everything is not automatically better.
  • The trend is coarser retrieval — bigger chunks, more of them, less reranking pressure — not the removal of retrieval.

  答题要点

  • 先区分「能不能塞进去」和「该不该每次都塞」，前者是技术问题，后者是成本问题。
  • 小规模、低频、少变的语料确实可以直接全塞，为它建检索系统是过度设计。
  • 检索不会被淘汰的三个理由：按次计费的成本、几十万篇塞不下的规模、必须在喂给模型之前完成的归因与权限过滤。
  • 材料越多，模型定位关键信息的稳定性越差，全塞不等于效果更好。
  • 趋势是检索粒度变粗——块更大、条数更多、重排压力变小，而不是检索消失。

D14 Capstone Project and Retrospective: A Multi-Tenant Enterprise Knowledge-Base Q&A, a RAG Decision Map, and an Interview Deep Dive

• You are handed a knowledge base of five million documents that must answer in about a second, with accuracy as the top priority. How would you design it?给你一个五百万文档、要求秒级响应、准确率优先的知识库场景，你会怎么设计这套系统？

  Common in ChinaCommon overseasDeep dive#system-design#scaling#latency-budget

  How to reason about it · think before answering

  1. The real subject here is not which technologies you know, it is whether you have a repeatable way to derive a configuration from constraints. Opening with an architecture diagram reads as a memorized answer; the way to score is to turn each constraint into a number first, then let every choice be forced by one of those numbers.
  2. Quantify the three constraints. Five million documents at roughly four or five chunks each is over twenty million chunks; at 1536 float dimensions that is hundreds of gigabytes, so the index does not fit in one machine's memory — that alone settles storage. A one-second budget to first token, with generation typically eating seven or eight hundred milliseconds, leaves only two or three hundred for retrieval. Accuracy first means you may trade latency and money for metrics, but only within that remaining budget.
  3. Now derive each knob from one of those numbers: a dedicated vector store or partitioning, plus half precision (its recall loss usually sits inside run-to-run noise while the index shrinks by about forty percent — essentially free); keep both keyword and vector routes with reciprocal rank fusion, because exact matches on document ids, error codes and names are a permanent blind spot for embeddings; rerank only the top twenty after fusion, since it buys ranking quality at the cost of one synchronous round trip, and a one-second budget affords exactly one.
  4. Then state two things you deliberately do not build, which is the part that reads as field experience. Agentic retrieval is not the default path: its gains concentrate on multi-hop questions while its cost is spread over every question, and it blows a one-second budget outright — the right move is a cheap classifier that routes only the multi-hop minority into the loop. Contextual chunk headers and similar tricks also wait, because they dilute the keyword route while helping the vector route; the directions are opposite, so measure on your own embeddings before committing.
  5. Accuracy first has to become something you can sign off on. That means a golden set of at least a hundred questions with multi-hop and unanswerable each above ten percent, recall and ranking quality read separately, abstention rate on unanswerable questions as its own column, and citations verified by code rather than trusted from the model. Reporting the ugliest column alongside the headline number is far more credible than reporting a single score.
  6. Expected follow-up: how do you build the first index over five million documents? It is a one-off large expense, so batch it, make it resumable, and put content-hash incremental sync in from day one, or every config change means buying the whole corpus again. Push further and you get to rollout: dual-write the new embeddings into a second column, evaluate both columns on the same golden set, then shift traffic, so rollback is a config flip rather than an eight-hour rebuild.

  分析过程 · 先想清楚再作答

  1. 这题的题眼不在「你会用什么技术」，而在「你有没有一套从约束推配置的方法」。开口就报架构图和技术栈的答案会被判成背方案；拿到分的答法是先把约束翻译成数字，再让每个选择被某个数字逼出来。
  2. 先把三个约束量化：五百万文档按一篇四五块估，是两千多万块，单精度 1536 维就是上百 GB，**索引塞不进单机内存**，这一条直接决定了存储选型；秒级响应意味着从收到问题到第一个字的预算大约一秒，而生成本身通常就吃掉七八百毫秒，检索侧只剩两三百毫秒；准确率优先意味着可以拿延迟和钱换指标，但只能换到那两三百毫秒为止。
  3. 然后逐项落地，每一项都挂在上面某个数字上：存储上专用向量库或分区加半精度量化（半精度的召回损失通常落在重跑噪声里，索引却小四成，这是白捡的）；检索保留关键词与向量两路加倒数排名融合，因为精确匹配的文档号、错误码、人名是向量的固定盲区；重排只作用于融合后的前二十条——它买的是排序质量，一次同步往返，秒级预算里放得下一次，放不下两次。
  4. 接着讲两个「不上」的决定，这一段比上面更能显出做过工程：**Agentic 检索不作为默认路径**，它的收益集中在多跳题上而代价摊给全部问题，秒级预算下更是直接超支——正确做法是先用一次便宜的分类把多跳分流出来，只让那一小部分进循环；**上下文块头之类的手法先不上**，因为它对关键词一路是稀释、对向量一路才是补位，方向相反，得在自己的真实 embedding 上测过再说。
  5. 准确率优先必须落成可验收的东西，否则是空话：一份不少于一百题的标准答案集（其中多跳与无答案各占一成以上）、召回率与排序质量分开看、无答案题的拒答率单独一栏、引用由代码回查而不是靠提示词自觉。**报数字时把最难看的那一栏也报出来**，比只报总分可信得多。
  6. 可预期的追问：五百万文档怎么建第一版索引？答案是这笔钱是一次性大额支出，要按批做、可断点续跑，并且从第一天就上基于内容指纹的增量同步——否则每次改配置都等于把整个知识库重买一遍。再追问就谈灰度：新旧两套向量双写在两列上，用同一份标准答案集在两列上各跑一遍再切流量，回滚只是改一个配置项。

  Key points

  • Translate constraints into numbers first: twenty million chunks means the index will not fit one machine, and a one-second budget leaves retrieval two to three hundred milliseconds.
  • Dedicated store or partitions plus half precision; keep keyword and vector routes with RRF, and rerank only the top twenty after fusion.
  • Name the two things you will not ship: agentic only for a routed multi-hop minority, and chunk headers only after measuring on your own embeddings.
  • Turn accuracy-first into a hundred-plus question golden set, abstention rate as its own column, and code-verified citations.
  • First index build is a one-off large expense: batch it, make it resumable, add incremental sync on day one, and dual-write columns for model swaps.

  答题要点

  • 先把约束翻译成数字：两千多万块决定索引塞不进单机内存，一秒预算里检索侧只剩两三百毫秒。
  • 存储用专用库或分区加半精度；检索保留关键词与向量两路加倒数排名融合，重排只作用于前二十条。
  • 明确说出「不上」的两项：Agentic 只对分流出来的多跳开，块头这类方向相反的手法先测再说。
  • 准确率优先要落成一百题以上的标准答案集、拒答率单独一栏、引用由代码回查。
  • 第一版建索引是一次性大额支出：分批可续跑，并从第一天就上增量同步；换模型走双写切列。

Build an AI Short-Drama Production Pipeline With Agents in 14 Days

D14 A Five-Episode Season: Batch Production, Portfolio Packaging, and a Short-Drama Pipeline Interview Deep Dive

• Walk me through the AI content pipeline you built. What was the hardest part?介绍一下你做的这条 AI 内容生产线，它最难的地方在哪？

  Common in ChinaCommon overseasBasic#project-storytelling#system-design

  How to reason about it · think before answering

  1. This is an open question that tests convergence. Narrating two weeks of work chronologically loses the interviewer in three minutes; delivering one through-line in thirty seconds is what counts as telling a project well.
  2. Open with positioning and scale: an automated pipeline from a one-line premise to publish-ready vertical episodes, one run producing a five-episode season, with humans stepping in only where judgement is required. Numbers first, detail second.
  3. Then answer hardest. That word should not be spent on debugging pain; spend it on a judgement that generates every downstream decision: video generation is the most expensive, slowest and most failure-prone stage at over ninety percent of per-episode cost, so the whole design revolves around issuing one fewer video call.
  4. Attach the chain of consequences in one sentence: idempotency and caching avoid duplicate calls, reference-image reuse reduces retries, the draft tier makes experimentation cheap, and the budget breaker stops a runaway. The chain proves your choices are derived rather than collected.
  5. Leave a deliberate hook for follow-up, such as saying the async task client turned out far harder than expected. That steers the interviewer toward your strongest material instead of a corner you never considered.
  6. Expect the follow-up: do you have real numbers? Keep four from every run: wall time, spend, failure rate and manual interventions. If spend is estimated, say so, rather than letting them assume you pasted a real invoice.

  分析过程 · 先想清楚再作答

  1. 这是一道开放题，考的是收敛能力。把十四天的东西按时间顺序流水账讲一遍，面试官三分钟后就走神了；能在三十秒内给出一条主线，才算会讲项目。
  2. 开头两句要立住定位与规模：从一句话选题到多平台可发布成片的自动化流水线，一次运行产出一季五集，人只在需要判断的地方介入。数字先给，细节后给。
  3. 然后回答「最难」。这个词不该答成「调试很麻烦」，要答成一条能推出后续所有设计的判断：这条线上最贵、最慢、最容易失败的是视频生成，占单集成本九成以上，所以整套工程都是围着「怎么少调一次视频接口」转的。
  4. 接着一句话挂上推论链：幂等与缓存是为了不重复调，参考图复用是为了少试几次，草稿档路由是为了试错时用便宜规格，预算熔断是为了失控时能停住。这条链子证明你的技术选择不是攒来的最佳实践。
  5. 最后主动留一个可被追问的钩子，比如「异步任务的客户端比我预想的复杂得多」——把面试官引到你准备最充分的地方去，而不是等他随机挑一个你没想过的角落。
  6. 可预期的追问是「有真实数据吗」。所以复盘时必须留下四个数字：耗时、花费、失败率、人工介入次数。花费是估算的就要主动说明是估算，别让人以为你贴了张真实账单。

  Key points

  • Position first: from a one-line premise to multi-platform episodes, one run per five-episode season
  • Frame the hardest part as a judgement: video dominates cost and is the slowest, most failure-prone stage
  • Show the derivation chain: idempotency and caching, reference reuse, draft tier, budget breaker
  • Bring four numbers: wall time, spend, failure rate, manual interventions, flagging estimates as estimates
  • Plant a follow-up hook that steers the conversation to your strongest area

  答题要点

  • 先定位再展开：从一句话到多平台成片，一次运行产出一季五集
  • 把最难点答成一条判断：视频占单集成本九成以上且最慢最易失败
  • 用推论链证明设计是导出来的：幂等缓存、参考图复用、草稿档、预算熔断
  • 带上四个数字：耗时、花费、失败率、人工介入次数，估算值要主动标注
  • 主动留一个追问钩子，把话题引向准备最充分的部分