Interview Bank

RAG in 14 Days: From Retrieval to Trustworthy Answers

D1 Why Retrieve at All: Hallucination, Knowledge Cutoffs, and the Cost of Long Context; a Minimal Keyword-Only RAG

• When should you use retrieval-augmented generation, when should you fine-tune, and when is stuffing the documents into the context window good enough?什么时候该用检索增强生成，什么时候该微调，什么时候直接把文档塞进上下文就够了？

  Common in ChinaCommon overseasBasic#rag-basics#fine-tuning#long-context

  How to reason about it · think before answering

  1. This question shows up in almost every loop. The differentiator is not reciting three definitions, it is offering a decision rule the interviewer can reuse.
  2. Lead with the rule: is the model missing knowledge, or missing a way of speaking? Missing knowledge means retrieval; missing style or output shape means fine-tuning. That single cut covers most cases.
  3. Then line up the three options against three costs: cost of updating knowledge, cost per request, and whether the answer can be traced back to a source. Retrieval updates by editing a file, fine-tuning takes a retraining cycle, and long-context pays for the whole corpus on every call.
  4. Give long-context its fair case: when the corpus is small, changes rarely, and request volume is low, stuffing it in is the cheapest engineering decision you can make. It stops being cheap once the corpus grows or the same material is queried thousands of times a day.
  5. Close by naming when none of this applies: if the answer does not depend on any external document (rewriting, translating, reformatting), retrieval only adds noise, latency and cost.
  6. Expected follow-up: can you do both? Yes, and it is common. Fine-tuning controls format and refusal behaviour, retrieval supplies the facts.

  分析过程 · 先想清楚再作答

  1. 这题几乎每场都问，区分度不在能不能背出三条定义，而在你会不会给一条判据。只说「RAG 适合动态知识、微调适合特定风格」的人一抓一大把，面试官等的是下一句。
  2. 先给一条能当场套用的判据：模型缺的是「知道什么」还是「怎么说」。缺知识走检索，缺风格与输出格式走微调，这一刀切下去能分掉八成场景。
  3. 再拿三笔账把三条路排开：知识更新的代价（改文件立刻生效 / 重训以天计 / 改文件立刻生效）、单次成本（只付取回的几段 / 只付推理 / 每次都付全量材料）、能不能归因（能 / 不能 / 能但材料一多定位会飘）。
  4. 把上下文直塞的适用边界说清楚：材料总量小、更新不频繁、对单次成本不敏感的场景它最划算，因为工程量近乎为零。一旦材料涨到几百篇，或者同一批材料每天要被问上万次，成本曲线立刻反超。
  5. 最后主动补一句「什么时候都不该用检索」——任务的答案不依赖任何外部文档时（改写、翻译、格式转换），加检索只会引入噪声、延迟和成本。能主动划出不该用的边界，比会背适用场景更能证明你做过。
  6. 可预期的追问：能不能既微调又检索？答案是可以，而且常见——微调管输出格式与拒答口径，检索管事实，两者解决的不是同一个问题。

  Key points

  • One rule: retrieval for missing knowledge, fine-tuning for a missing way of speaking.
  • Retrieval updates instantly by editing files, supports citation, and costs scale with the retrieved passages rather than the corpus.
  • Fine-tuning is good at locking in style and output schema, poor at loading facts, and offers no traceability.
  • Long-context stuffing wins when the corpus is small, stable and queried infrequently; it loses on cost and on locating facts once the corpus grows.
  • If the answer does not depend on any document, use none of them.

  答题要点

  • 一条判据：缺「知道什么」用检索，缺「怎么说」用微调。
  • 检索改文件即时生效、可归因、成本只跟取回的几段有关，代价是要自己建一套会出错的检索系统。
  • 微调擅长固化风格与输出格式，不擅长灌事实：数据一变就要重训，而且没法归因。
  • 长上下文直塞在小型、低频、少变的语料上最划算，材料变多或调用量变大之后成本与定位稳定性都会恶化。
  • 任务答案不依赖外部文档时三条路都不该用，直接调模型。