Interview Bank

RAG in 14 Days: From Retrieval to Trustworthy Answers

D6 The Generation Side: Ordering Context, Labeling Citations, When You Must Refuse to Answer, and Streaming Responses

• Does the ordering of retrieved passages in the context affect answer quality? If so, how would you order them?上下文里材料的排列顺序会影响回答质量吗？如果会，你会怎么排？

  Common in ChinaCommon overseasBasic#context-assembly#prompt-engineering#ordering

  How to reason about it · think before answering

  1. This is a warm-up question, but 'sort by relevance descending' only earns half the credit. The interviewer wants to know whether you treat position itself as a variable.
  2. State the conclusion first: it does matter. Models attend more reliably to material at the start and the end of the context, and are most likely to miss what sits in the middle. Plain descending order therefore parks your second-best passage in the worst spot.
  3. Give the ordering: rank one first, rank two last, rank three second, rank four second-to-last, folding inward. Whatever ends up in the middle is by construction the least important, so the cost of it being skipped is smallest.
  4. Round it out with the other assembly steps, which shows you have written this code: a deterministic tiebreaker (otherwise block numbers drift between runs and your logs stop matching), dedupe on normalised text, and a token budget that skips rather than stops when a block does not fit.
  5. Expected follow-up: how would you verify this? Do not guess. Hold the question set fixed, vary only the ordering, and measure. Position effects differ by model and context length, so treat it as a parameter to measure on your own data rather than a universal law.

  分析过程 · 先想清楚再作答

  1. 这是一道送分题，但答成「按相关性从高到低排」就只拿到一半分。面试官想听的是你知不知道位置本身是个变量。
  2. 结论先说：会影响。模型对上下文开头和结尾的材料明显更敏感，正中间的最容易被读漏。所以简单按分数从高到低顺排，等于把第二重要的材料放进了最不容易被读到的位置。
  3. 给出排法：第 1 名放开头、第 2 名放结尾、第 3 名放第二位、第 4 名放倒数第二位，依次往里收。这样按分数排下来越靠中间的块本来就越不重要，被读漏的代价最小。
  4. 顺带把排序之外的三道手续说全，显得你真的写过这段代码：同分要有决胜键（否则块编号会在两次运行之间飘，日志对不上）、要按归一化文本去重（同一段话常在手册和问答里各出现一次）、要有 token 预算并且塞不下时不要直接停。
  5. 可预期的追问：这个结论怎么验证？答案是别猜——固定一批问题，只改排列顺序跑对照，看指标差多少。位置效应在不同模型、不同上下文长度上强弱不一样，把它当成一个要在自己数据上量的参数，而不是一条普适定律。

  Key points

  • Yes: material at the head and tail is used more reliably, the middle is most often skipped.
  • Put the strongest at both ends: rank one first, rank two last, rank three second, folding inward.
  • Assembly also needs a deterministic tiebreaker for stable numbering, dedupe on normalised text, and a token budget that skips oversized blocks instead of stopping.
  • The strength of the effect varies by model and context length, so measure it on your own data instead of quoting it as a law.

  答题要点

  • 会影响：开头和结尾的材料更容易被用上，正中间的最容易被读漏。
  • 排法是最重要的放两端：第 1 名开头、第 2 名结尾、第 3 名第二位，依次往里收。
  • 组装还要做三件事：同分给决胜键保证编号稳定、按归一化文本去重、控 token 预算且塞不下时跳过而不是终止。
  • 位置效应的强弱因模型与上下文长度而异，要在自己的数据上做对照实验量出来，不能当普适定律照搬。