面试题库

14 天 RAG：从检索到可信回答

D12 Agentic RAG：把检索做成工具，让模型自己决定查不查、查几次、要不要推翻重来

• 把检索包成一个工具交给模型，这个工具的描述该怎么写？写不好会导致哪些具体的错误行为？You are exposing retrieval to a model as a tool. How do you write the tool description, and what concrete failure modes appear when you write it badly?

  国内高频海外高频基础#tool-design#agentic-rag#prompting

  分析过程 · 先想清楚再作答

  1. 这题的题眼是「具体的错误行为」。只会背「描述要写清楚工具的用途」的，一句话就暴露了没上过线——面试官想听的是描述里少一句话，线上就多一类工单。
  2. 先给结构：一段合格的工具描述要回答四件事——库里有什么和没有什么、什么时候必须用、什么时候不要用、查询串写成什么形状。四条各对应一类事故，逐条挂钩着说最有说服力。
  3. 逐条挂钩：不写范围，模型拿它当搜索引擎，问天气也去查；不写「必须用」，涉及公司制度的问题被模型凭记忆编答案，而且编得非常像真的；不写「不要用」，闲聊和翻译都触发一次无谓检索，成本和延迟白涨；不写查询形状，模型把用户整句问话塞进 query，「叫什么名字」这种疑问词进了检索，纯噪声。
  4. 最后一条最值钱也最容易漏：在描述里加一句「写成关键词短语，不要带疑问词」，比在检索侧做十种查询清洗都管用——问题在源头，就在源头修。
  5. 补一个生产视角：参数里的过滤字段（比如部门）要写明「只在确定时才填」。模型倾向于把可选参数填满，填错一个部门就把正确答案挡在库外，而这种错误在日志里看不出来，表现是「检索没结果」。
  6. 可预期的追问是「怎么验证描述写对了」。答案是拿一批负样本跑：闲聊、翻译、算术、以及答案已在对话里的追问，看模型有没有多调一次工具；这类回归是能自动化的。

  How to reason about it · think before answering

  1. The discriminator is whether you can name concrete failure modes. Reciting 'the description should be clear' signals you have never shipped one.
  2. Give the structure first: a usable description answers four things - what is and is not in the corpus, when the tool must be called, when it must not be called, and what shape the query string should take.
  3. Attach a failure to each: no scope and the model treats it as a web search; no 'must call' and it answers policy questions from memory, convincingly; no 'must not call' and greetings or translations each burn a retrieval; no query shape and the model pastes the raw user sentence in, dragging interrogative words into the index.
  4. The query-shape line is the cheapest win: one sentence saying 'keyword phrase, no question words' beats ten heuristics for query cleaning on the retrieval side.
  5. Production angle: optional filter parameters such as department need an explicit 'only set this when you are certain'. Models like to fill optional fields, and a wrong filter hides the correct answer while the logs only show 'no results'.
  6. Expected follow-up: how do you verify the description works? Run a negative suite - small talk, translation, arithmetic, follow-ups already answered in the conversation - and assert the tool was not called. That regression is automatable.

  答题要点

  • 描述是写给模型看的提示词，不是注释；四段式：范围、什么时候用、什么时候不用、查询写成什么形状。
  • 不写范围会被当成搜索引擎；不写「必须用」会导致凭记忆编答案。
  • 不写「不要用」会让闲聊也触发检索，成本和延迟白涨。
  • 写明查询要用关键词短语、不带疑问词，比在检索侧清洗查询更根本。
  • 可选过滤参数要写「只在确定时才填」，填错会静默地把正确答案挡在外面。
  • 用一批负样本（闲聊、翻译、算术）做回归，断言工具没有被调用。

  Key points

  • The description is a prompt for the model, not a code comment: scope, when to call, when not to call, query shape.
  • Missing scope turns it into a web search; missing 'must call' produces confident answers from memory.
  • Missing 'do not call' makes small talk trigger retrieval, paying cost and latency for nothing.
  • Stating 'keyword phrase, no question words' fixes query pollution at the source.
  • Optional filters need 'only set when certain' - a wrong filter silently hides the right answer.
  • Regression-test with a negative suite and assert the tool was not invoked.