面试题库

14 天 RAG：从检索到可信回答

D10 查询侧优化：改写、假设文档嵌入、多路查询、后退提问与意图路由

• 意图路由判错了会怎样？你会怎么设计兜底？What happens when intent routing misclassifies, and how would you design the fallback?

  国内高频海外高频进阶#intent-routing#fallback#observability

  分析过程 · 先想清楚再作答

  1. 这题在考「有没有想过错误的方向」。路由是分类器，分类器一定会错；只答「多加训练数据提高准确率」的，等于没回答兜底怎么设计。
  2. 先把错误按方向拆开，这一步是整题的骨架：三条路（直接回答、单跳检索、多跳检索）两两误判，代价完全不对称。把该检索的判成直接回答，模型手里一点材料都没有，只能编，这是最贵的一种错；把闲聊判成单跳，只是白花一次检索；把多跳判成单跳，只是少查一轮、答得不全。
  3. 结论顺势就出来了：**兜底方向要偏向「多花一点钱」，判不出来一律退回单跳检索。** 单跳是三条路里错得最轻的一条，而且它的错误是可恢复的——材料不全模型还能说「资料里只查到一半」，材料为空它就只能编。
  4. 再补一层运行时兜底，比事前分类更管用：分类成直接回答之后，如果模型的回答里出现了具体数字、金额、日期这类需要出处的内容，就回退去检索一次再答；分类成单跳之后，如果检索侧一条都没过门槛，就升级走多跳或直接拒答。**用后一步的观测结果纠正前一步的判断**，这是路由系统最实用的一条设计。
  5. 还要提一句可观测性：路由的每一次判定都要落日志，带上原始问题、判定结果、后续是否发生了兜底升级。没有这份日志，你既不知道路由准不准，也没法攒出下一版的训练集。
  6. 可预期的追问是「什么时候干脆别做路由」。答：流量里闲聊占比很低、且多跳问题很少时，路由省下的钱还不够付分类调用的钱，这时候直接全部走单跳更划算——我们在 30 篇语料的实验里就看到，路由真正的收益并不在省检索，而在于认出多跳之后给它更高的上下文预算。

  How to reason about it · think before answering

  1. This tests whether you have thought about the direction of the error. A router is a classifier and classifiers misfire; "add more training data" is not a fallback design.
  2. Break the errors down by direction — that is the backbone of the answer. Across three routes (direct answer, single-hop, multi-hop) the six confusions carry wildly asymmetric costs. Routing a retrieval-worthy question to a direct answer leaves the model with no material at all, so it fabricates: the most expensive error. Routing chit-chat to single-hop merely wastes one retrieval. Routing multi-hop to single-hop just yields an incomplete answer.
  3. The conclusion follows: bias the fallback toward spending a little more, and default to single-hop retrieval whenever the classifier is unsure. Single-hop is the cheapest error to make, and it is recoverable — with partial material the model can still say it only found half the answer; with no material it can only invent one.
  4. Add a runtime fallback, which beats better up-front classification: after a direct-answer routing, if the draft reply contains figures, amounts or dates that need a source, fall back to retrieval and answer again; after a single-hop routing, if no candidate clears the admission gate, escalate to multi-hop or abstain. Correcting the earlier decision with the later observation is the single most useful pattern in routing systems.
  5. Mention observability: log every routing decision with the raw question, the label, and whether a fallback fired. Without that log you know neither how accurate the router is nor what to train the next version on.
  6. Expect "when should you skip routing entirely?" When chit-chat is a small share of traffic and multi-hop questions are rare, the classification call costs more than it saves. In our 30-document lab the real gain from routing was not saved retrievals but the ability to give recognised multi-hop questions a larger context budget.

  答题要点

  • 三条路的误判代价不对称：把该检索的判成直接回答最贵（模型没材料只能编），把闲聊判成单跳只是白花一次检索。
  • 兜底方向偏向多花钱：判不出来一律退回单跳检索，它是错得最轻且可恢复的一条路。
  • 加运行时兜底：直接回答里出现需要出处的数字就补一次检索；单跳检索一条都没过门槛就升级或拒答。
  • 每一次路由判定都落日志（原始问题、判定结果、是否触发兜底），既用于监控也用于攒下一版训练集。
  • 闲聊与多跳占比都很低时，路由省的钱付不起分类调用，直接全走单跳更划算。

  Key points

  • The three routes have asymmetric error costs: sending a retrieval-worthy question to a direct answer is the worst, while routing chit-chat to single-hop only wastes one retrieval.
  • Bias the fallback toward spending more: default to single-hop whenever the classifier is unsure, since that error is the mildest and is recoverable.
  • Add runtime fallbacks: re-retrieve if a direct answer contains figures that need a source; escalate or abstain if no single-hop candidate clears the gate.
  • Log every routing decision — raw question, label, whether a fallback fired — for both monitoring and the next training set.
  • When chit-chat and multi-hop are both rare, the classification call costs more than it saves; route everything to single-hop instead.

D14 综合项目与复盘：多租户企业知识库问答，一张 RAG 决策地图与面试专题

• RAG 系统上线后用户反馈「答得不准」，你的排查顺序是什么？Users report that your live RAG system 'answers inaccurately'. What is your triage order?

  国内高频海外高频进阶#debugging#failure-modes#observability

  分析过程 · 先想清楚再作答

  1. 这题几乎是必考题，而绝大多数人答成一堆并列的可能性：可能是切块问题、可能是提示词问题、可能是模型不行。并列不是排查，排查的意思是**有顺序、有判据、每一步能把可能性砍掉一半**。
  2. 先把「答得不准」这四个字拆开——它至少塞了四种病，而且修法互不通用：答非所问、只答得出片段、引用错位、更新不生效。所以第一个动作不是改配置，是**拿到具体的问题和回答，把它归到这四类里的一类**。
  3. 然后给顺序，而且要说清顺序的理由：**排查从右往左看、修复从左往右修**。从右往左是因为你最先看到的是生成结果；从左往右是因为上游的错会被下游放大——检索没捞到的东西，再好的提示词也救不回来。具体走法是：打印这一问的候选池和最终上下文，先看答案文档在不在候选池里。不在，是检索的债；在候选池但没过准入门槛，是门槛的债；过了门槛却没装进上下文预算，是块太大或预算太小；都进了而模型没用上，才轮到生成侧。
  4. 这里有一个容易写错的细节值得主动讲：**多跳题的诊断对象是缺的那几篇，不是「有没有捞到任意一篇」**。我们实验里有一道题要同时命中两篇，第一篇稳稳排第一、第二篇一次都没进候选池；用「任意一篇」去判会把它归成预算问题，然后你去调预算，调一整天也没用。这一条区分度很高，因为它只有真的按题排查过才想得到。
  5. 第四类「更新不生效」发生在问答之外，判据是另一条：先看对账认没认出这篇改了（内容指纹算之前有没有做换行归一化），再看缓存的 key 里有没有把索引版本和权限范围算进去。「什么时候必须失效」等价于「key 里有没有把那样东西算进去」，key 少放一样，那样东西变了缓存就不会失效。
  6. 可预期的追问：怎么让这套排查不靠人肉？答案是把分类做进评估面板——每一道没中的题自动标出它属于四类中的哪一类，并按租户分开统计。全局平均会把单个客户的塌方按人头摊薄，而线上会投诉的恰恰是那个客户。

  How to reason about it · think before answering

  1. This one is almost guaranteed to be asked, and most people answer with a flat list of possibilities: maybe chunking, maybe the prompt, maybe the model. A list is not triage. Triage means an order, a decision rule at each step, and each step eliminating half the search space.
  2. First decompose the complaint. 'Inaccurate' hides at least four distinct failures whose fixes do not transfer: off-topic answers, partial answers, misaligned citations, and stale content. So the first action is not to change a setting, it is to obtain the specific question and answer and classify it into one of those four.
  3. Then give the order along with its justification: read the pipeline right to left, fix it left to right. Right to left because the generated answer is what you see first; left to right because upstream errors are amplified downstream — no prompt can recover a document retrieval never fetched. Concretely: dump the candidate pool and the final context for that question, and check whether the answer document is in the pool at all. Absent means a retrieval debt; present but below the admission gate means a gate debt; admitted but never packed into the context budget means chunks too large or budget too small; all present and still unused means it is finally a generation problem.
  4. One detail worth volunteering because it is easy to get wrong: for multi-hop questions, diagnose the documents that are missing, not whether any one of them was retrieved. In our experiment one question needed two documents; the first ranked first every time and the second never entered the candidate pool at all. Judging by 'any of them' labels it a budget problem, and you can spend a full day tuning budgets to no effect. This distinction only occurs to someone who has actually triaged question by question.
  5. The fourth class, stale content, happens outside the question path and has its own rule: first check whether reconciliation even noticed the edit (was the content hash computed after line-ending normalization?), then check whether the cache key includes the index version and the permission scope. 'When must this expire' is equivalent to 'is that thing part of the key' — leave something out of the key and changes to it will never invalidate the entry.
  6. Expected follow-up: how do you stop relying on manual triage? Build the classification into the evaluation panel so every missed question is automatically labelled with one of the four classes, and report it per tenant. A global average dilutes one customer's collapse across the whole population, and that customer is exactly the one who will file the complaint.

  答题要点

  • 先把「答得不准」归类成四种病：答非所问、只答得出片段、引用错位、更新不生效——修法互不通用。
  • 排查从右往左看、修复从左往右修：先打印候选池与最终上下文，看答案文档卡在哪一层。
  • 四层判据依次是：没进候选池、进了没过门槛、过了没装进预算、都进了模型没用上。
  • 多跳题只诊断缺的那几篇；用「有没有捞到任意一篇」会把「根本没捞到」误判成预算问题。
  • 「更新不生效」查对账与缓存 key：什么时候必须失效，等价于 key 里有没有算进那样东西。

  Key points

  • Classify the complaint into four failures first — off-topic, partial, misaligned citation, stale — because their fixes do not transfer.
  • Read right to left, fix left to right: dump the candidate pool and final context and find which layer the answer document stalls at.
  • The four rules in order: never retrieved, retrieved but below the gate, admitted but squeezed out of the budget, packed but unused by the model.
  • For multi-hop, diagnose only the missing documents; judging by 'any one retrieved' mislabels a never-retrieved case as a budget problem.
  • For stale content, check reconciliation and the cache key: what must expire is exactly what the key must contain.