Interview Bank

RAG in 14 Days: From Retrieval to Trustworthy Answers

D10 Query-Side Optimization: Rewriting, Hypothetical Document Embeddings, Multi-Query, Step-Back Prompting, and Intent Routing

• How do you handle coreference in multi-turn RAG, and what is the classic failure when you skip it?多轮对话里怎么处理指代？不做指代消解最典型的翻车场景是什么？

  Common in ChinaCommon overseasBasic#coreference#multi-turn#query-rewriting

  How to reason about it · think before answering

  1. This is a warm-up question, but there is still a gap between answers. Saying "just concatenate the history into the query" invites a follow-up about growing histories that most candidates cannot handle.
  2. State the mechanism: insert a short rewrite call before retrieval that takes the last few turns plus the current question and returns one retrieval-ready line. Set temperature to 0 so the same input always yields the same query, and forbid the model from answering the question in the prompt.
  3. Explain why concatenation is worse: history grows without bound, filler words dilute inverse document frequency, and the previous answer leaks in — you end up retrieving an answer with an answer. The rewriter emits one sentence, not a transcript.
  4. Make the failure concrete. Turn one: "who must sign off on this operation?" Answer: "the platform team lead." Turn two: "what is that person's name?" Retrieved unresolved, not a single candidate clears the admission gate and the system refuses — even though the corpus contains the answer. The failure is not a wrong answer, it is a false "not found" right after the user's own question.
  5. Add the ordering trap: rewrite before intent routing. A pronoun is a classic multi-hop signal, so an unresolved query gets routed to the expensive path for nothing; after rewriting it is an ordinary single-hop question. Multi-query and step-back must also sit downstream of the rewrite, or one unresolved pronoun becomes three.
  6. Expect "how do you decide when to rewrite?" Trigger on short queries, pronouns and elliptical follow-ups; skip on a clearly new topic. The check is nearly free and removes most of the calls.

  分析过程 · 先想清楚再作答

  1. 这是一道送分题，但送分题也有高下之分：只说「把历史拼进查询里」的答案，会被追问一句「历史越拼越长怎么办」就卡住。
  2. 先把做法说清：在检索之前加一次很短的改写调用，输入是最近几轮对话加本轮问题，输出是一行可以直接检索的检索式；温度设 0 保证同一句话每次改成同一个结果，并在提示词里明确禁止模型顺手回答问题。
  3. 为什么不是「把历史整个拼进查询」：历史越拼越长，噪声词把逆文档频率摊薄，检索反而更差；而且历史里包含上一轮的答案，等于拿答案去检索答案。改写的产出是一句话，不是一段历史。
  4. 最典型的翻车场景要举实例：上一轮问「这个操作必须由谁审批」，答「必须由某某组组长审批」；这一轮问「这个人叫什么名字」。不消解直接检索这七个字，实测是**一条候选都过不了门槛，系统只能拒答**。注意失败方式不是答错，是「明明语料里有答案却说找不到」，用户体验是崩塌式的。
  5. 补一条顺序上的坑：改写必须在意图路由**之前**。「这个人」是典型的多跳信号词，路由看到它会判成多跳、白跑一轮；改写之后它只是个普通单跳问题。同理，多路查询、后退提问也都要建立在改写后的那句话上，否则错误被放大好几倍。
  6. 可预期的追问是「怎么知道要不要改写」。答：短问题、含指代词、含省略（「那审计日志呢」）时才触发，纯新话题跳过——这一步很便宜，但能省掉一大半调用。

  Key points

  • Add a short rewrite call before retrieval: last few turns plus current question in, one retrieval line out, temperature 0, answering explicitly forbidden.
  • Do not splice the whole history into the query — it grows unbounded, dilutes IDF, and leaks the previous answer into the search.
  • Classic failure: an unresolved pronoun means no candidate clears the gate, so the system refuses a question the corpus can answer.
  • That false "not found" hurts more than a wrong answer, since the user just asked about the same thing.
  • Order matters: rewrite first, then route; multi-query and step-back both build on the rewritten query.

  答题要点

  • 在检索前加一次短改写调用，输入最近几轮加本轮问题，输出一行检索式，温度 0，禁止模型回答问题。
  • 不要把历史整段拼进查询：越拼越长、噪声稀释逆文档频率，还会拿上一轮的答案去检索。
  • 典型翻车：上一轮的「这个人 / 他 / 那个」不消解，检索一条都过不了门槛，系统在有答案的情况下拒答。
  • 失败方式是「假的查不到」，比答错更伤体验，因为用户刚刚才问过同一件事。
  • 顺序：先改写、再路由，多路查询与后退提问都建立在改写后的查询上。
• Query rewriting adds a model call per question and doubles end-to-end latency. How do you decide whether it is worth paying?上线查询改写之后每问多了一次模型调用，端到端延迟涨了一倍，你怎么判断这笔开销值不值？

  Common in ChinaCommon overseasDeep dive#cost-tradeoff#latency#query-rewriting

  How to reason about it · think before answering

  1. This question is about turning an engineering judgement into numbers. "Rewriting obviously helps quality" is a fail — the candidate never measured the gain.
  2. Start with one question that nearly settles it: which slice of traffic does the gain land on? Query rewriting buys almost nothing on single-turn questions (we measured identical metrics with it on and off across 20 single-turn items); the entire payoff is in follow-up turns. So step one is to pull the share of multi-turn sessions from production logs.
  3. Step two is to lay out all three ledgers, because one alone cannot support a decision: how much the metrics moved on a fixed golden set, how much latency grew (a rewrite is a short-output task, so a cheap fast model often costs a few hundred milliseconds rather than doubling anything), and how many extra calls were added — one model call for rewriting versus one call plus several retrievals for multi-query is a completely different cost shape.
  4. Step three is to price the cheaper variants before deciding: rewrite only when a trigger fires (short query, pronoun, ellipsis), cache rewrites per session, and run a small model instead of the main one. These usually remove most of the cost while keeping the gain.
  5. Land on a usable rule: gain times affected traffic share, divided by added latency and cost, ranked against your other candidate optimisations. Rewriting usually ranks high because its failure mode is a false "not found" immediately after the user's own question — an abandonment-grade experience bug, not a few metric points.
  6. Expect "what if the latency genuinely is unacceptable?" Fire the rewrite and the first retrieval in parallel: search with the raw query immediately, search again when the rewrite returns, and fuse both rankings. You pay a max instead of a sum, at the cost of one extra retrieval.

  分析过程 · 先想清楚再作答

  1. 这题考的是「能不能把工程判断落到数字上」。凡是回答「改写当然要做，能提升效果」的，一律判为没做过——他连收益是多少都没量。
  2. 先问自己一句：**收益出现在哪一类流量上**。这一条几乎决定了答案。查询改写在单轮问答上的收益接近零（我们在 20 道单轮题上实测开关它指标一模一样），收益全在多轮追问。所以第一步是去线上日志里查多轮会话占比，占比很低的话这笔钱不该花在全量流量上。
  3. 第二步是把三笔账摆齐，缺一笔就不能下判断：指标涨了多少（用固定的标准答案集跑，不要用感觉）、延迟涨了多少（改写是短输出任务，可以换便宜快的那一档模型，往往只多两三百毫秒而不是翻倍）、多了几次调用（改写是一次，多路查询是一次调用加几次检索，成本结构完全不同，别混着算）。
  4. 第三步是找**便宜的替代路径**再比一次：只在命中触发条件时才改写（短问题、含指代词、含省略），纯新话题直接跳过；改写结果按会话缓存；用小模型跑改写而不是主模型。这三招通常能把这笔开销压掉一大半，而收益几乎不掉。
  5. 结论要落成一条可执行的判据：**收益乘以受影响流量占比，除以增加的延迟与成本**，跟你手上其他候选优化排个序。改写通常能排到很前面，因为它的失败方式是「用户明明追问同一件事却被告知查不到」，那是会直接导致弃用的体验故障，不只是指标掉几个点。
  6. 可预期的追问是「延迟真的不能接受怎么办」。答：把改写和第一次检索**并行发**，用原查询先检索一路，改写回来后再补一路，两路用倒数排名融合合起来——延迟只多一个 max 而不是一个加法，代价是多一次检索。

  Key points

  • Locate the gain first: rewriting is near-zero on single-turn traffic and pays off on follow-ups, so start from the share of multi-turn sessions.
  • All three ledgers are mandatory: metric delta on a golden set, added latency, added calls and token cost.
  • Try the cheap variants before deciding: conditional triggering, per-session caching, and a small model for the rewrite.
  • Decide on gain times affected traffic share over added latency and cost, then rank it against your other optimisations.
  • If latency is a hard constraint, fire the rewrite in parallel with the first retrieval and fuse both rankings, turning a sum into a max.

  答题要点

  • 先定位收益落在哪一类流量：改写在单轮上接近零收益，价值全在多轮追问，先查多轮会话占比。
  • 三笔账缺一不可：标准答案集上的指标变化、增加的延迟、增加的调用次数与 token 成本。
  • 先试便宜的替代路径：条件触发、按会话缓存、用小模型跑改写，通常能压掉大半开销。
  • 判据是「收益 × 受影响流量占比 ÷ 增加的延迟与成本」，再和其他候选优化排序。
  • 延迟真的卡死时，把改写与首次检索并行发，两路名次用倒数排名融合，延迟从加法变成取最大值。