Interview Bank

RAG in 14 Days: From Retrieval to Trustworthy Answers

D1 Why Retrieve at All: Hallucination, Knowledge Cutoffs, and the Cost of Long Context; a Minimal Keyword-Only RAG

• In BM25, what problems do term-frequency saturation and document length normalisation each solve? What happens if you set both k1 and b to zero?BM25 里的词频饱和与文档长度归一化分别在解决什么问题？把 k1 和 b 都设成 0 会发生什么？

  Common in ChinaCommon overseasIntermediate#bm25#ranking#information-retrieval

  How to reason about it · think before answering

  1. This checks whether you have actually read the formula rather than merely called a library. The test is whether you can map k1 and b onto specific terms and name the failure each one prevents.
  2. Start with the two holes in raw term frequency: keyword stuffing lets one document dominate by repeating a word, and long documents win by accident because they contain more words overall.
  3. k1 closes the first hole. Term frequency appears in both numerator and denominator, so the ratio approaches a ceiling instead of growing linearly. Fifty mentions are more relevant than five, but not ten times more relevant. A smaller k1 saturates sooner.
  4. b closes the second. The normalisation factor is one minus b plus b times document length over average length: at b equal to zero length is ignored entirely, at one it is fully penalised, and 0.75 is the conventional compromise.
  5. Now the trap in the question: k1 equal to zero collapses the ratio to a constant, so one occurrence scores the same as a hundred and matching becomes boolean. b equal to zero removes length entirely. Set both to zero and BM25 degenerates into a plain sum of inverse document frequencies.
  6. Expected follow-up: can you drop the IDF term? No. Without it, ubiquitous words drown everything else, and it is precisely IDF that lets BM25 work without a stopword list.

  分析过程 · 先想清楚再作答

  1. 这题考的是你有没有真的读过公式，而不是有没有调过库。判据很明确：能不能把 k1 和 b 各自对应到公式里的哪一项，并说出去掉之后会被什么样的文档钻空子。
  2. 先说朴素词频的两个漏洞：一是重复刷词，一篇文章把关键词写五十遍就能霸榜；二是长文占便宜，文档越长越容易蒙中查询里的词。这两个漏洞正好对应两个修正。
  3. k1 管第一个漏洞。分子分母里都有词频 f，所以词频涨上去之后整个分式趋近一个上界而不是线性增长——写五十遍确实比写五遍相关，但绝不该相关十倍。k1 越小饱和越快。
  4. b 管第二个漏洞。归一化项是 1 减 b 加上 b 乘以本文长度除以平均长度，b 等于 0 时完全不看长度，b 等于 1 时完全按长度比例惩罚，0.75 是长期折中的默认值。
  5. 回到题干那个陷阱：k1 设成 0 会让分式退化成常数，词出现一次和一百次得分完全一样，等于只剩「有没有出现过」的布尔匹配；b 设成 0 则长度信息彻底消失。两个一起设成 0，BM25 就退化成对逆文档频率求和，跟词频再无关系。
  6. 可预期的追问：那逆文档频率去掉行不行？答案是不行，去掉之后「的」「我们」这类高频词会淹没一切——而且要顺带说明 BM25 因此天然不需要停用词表，这一句最能体现你读懂了公式。

  Key points

  • k1 controls saturation and prevents keyword stuffing: the score approaches a ceiling rather than growing linearly with frequency.
  • b controls length normalisation and stops long documents from winning by sheer word count.
  • Setting k1 to zero degenerates the scorer into boolean matching; one occurrence scores the same as a hundred.
  • Setting b to zero removes document length from the equation entirely; both at zero leaves only a sum of IDF terms.
  • IDF is the third component: it up-weights rare terms and removes the need for a stopword list.

  答题要点

  • 词频饱和由 k1 控制，防的是重复刷词：词频涨大后得分趋近上界而非线性增长。
  • 长度归一化由 b 控制，防的是长文档靠词多蒙中查询，用本文长度比平均长度把它压回去。
  • k1 设 0 会退化成布尔匹配，词出现一次和一百次同分；b 设 0 则完全不考虑文档长度。
  • 两者都设 0 时 BM25 只剩逆文档频率求和，等于放弃了词频信息。
  • 逆文档频率是第三块，让稀有词权重更高，也让 BM25 天然不需要停用词表。