面试题库
共 328 题,当前筛选 6 题。
课程全部30 天从前端工程师到 Agent 工程师5 天提示词工程零基础Claude 高效使用:从对话到 Claude CodeCodex 与 OpenAI Agents SDK 高效使用7 天 MCP:把工具接进任何 Agent7 天 Agent Skills:把经验做成可复用能力5 天上下文工程14 天 RAG:从检索到可信回答14 天用 Agent 搭一条 AI 短剧生产线
标签
全部#tool-design6#evaluation30#cost29#reliability24#architecture22#agent-skills21#security18#observability15#streaming12#error-handling11#idempotency10#system-design10
还有 362 个标签收起标签
#api-design9#coding-agent9#distributed-systems8#multi-agent8#rag8#chunking7#debugging7#pipeline-design7#structured-output7#agent-loop6#mcp6#operations6#prompt-injection6#sse6#tool-calling6#context5#context-engineering5#deployment5#embeddings5#hybrid-search5#message-bus5#scheduling5#system-prompt5#agentic-rag4#behavioral4#client4#concurrency4#consistency4#framework-design4#ingestion4#prompt-engineering4#rate-limiting4#retrieval4#routing4#trade-offs4#abstention3#agents-sdk3#caching3#communication3#context-window3#cost-control3#data-quality3#failure-modes3#image-generation3#interview-prep3#langgraph3#latency3#llm-as-judge3#llm-basics3#long-context3#model-migration3#model-routing3#orchestration3#ordering3#progressive-disclosure3#prompt-basics3#prompt-caching3#protocol3#provider-abstraction3#ranking3#recall3#redis-streams3#responses-api3#resume3#retry3#runtime3#scalability3#skills3#state-management3#statelessness3#subagents3#timeline3#versioning3#workflow-engine3#access-control2#agent-design2#async-task2#auth2#checkpointing2#citation-verification2#citations2#claude-md2#code-review2#compaction2#context-rot2#contextual-retrieval2#cost-tradeoff2#data-modeling2#database2#distribution2#fallback2#ffmpeg2#golden-set2#grounding2#interview-process2#long-term-memory2#media-pipeline2#memory2#multi-hop2#multi-tenancy2#oauth2#prioritization2#project-storytelling2#prompt-techniques2#query-rewriting2#react2#refusal2#reproducibility2#rerank2#retrieval-quality2#retrospective2#scripts2#sharding2#state-machine2#tool-permissions2#tooling2#tools2#tracing2#transport2#tts2#ux2#vector-database2#verification2#workflow2#agent-basics1#agent-quality1#agent-sdk1#agents-md1#altitude1#analytics1#approvals1#architecture-review1#async1#atomicity1#attention-budget1#av-sync1#backoff1#bi-encoder1#bm251#budget-control1#build-vs-buy1#cancellation1#candidate-selection1#capacity-planning1#career1#chain-of-thought1#ci1#circuit-breaker1#claude-code1#cli-design1#client-integration1#cloud1#compliance1#compression1#configuration1#confused-deputy1#consistent-hashing1#content-hash1#content-safety1#context-assembly1#context-compression1#context-management1#copyright1#coreference1#correctness1#cost-accounting1#cost-analysis1#cost-optimization1#cross-encoder1#cross-model1#customer-support1#dag1#decision-making1#decomposition1#degradation1#deliberate-practice1#design1#diagnostics1#dimensions1#docker1#documentation1#embedding-migration1#encoding1#engineering-judgement1#engineering-tradeoffs1#error-propagation1#escalation1#eval1#event-driven1#evidence1#failure-analysis1#fairness1#faithfulness1#fan-out1#feedback-loop1#fencing-token1#few-shot1#filter-pushdown1#filtering1#fine-tuning1#forking1#four-elements1#framework-selection1#frontend1#global-market1#graph-rag1#guardrails1#hallucination1#handoff1#handoffs1#headless1#hnsw1#hooks1#human-in-the-loop1#hybrid1#hyde1#incremental-recompute1#incremental-sync1#index-maintenance1#index-routing1#indexing1#information-retrieval1#instruction-hierarchy1#integration1#intent-routing1#interrupt-merge1#invalidation1#isolation1#iterative-scan1#ivfflat1#json-parsing1#json-schema1#just-in-time1#jwt1#knowledge-organization1#labeling1#latency-budget1#lease1#least-privilege1#llm-output-quality1#long-session1#loop-guard1#maintenance1#mcp-basics1#mental-model1#messages-api1#metadata1#methodology1#metrics1#migration1#mobile1#model-selection1#moderation1#modularity1#multi-turn1#multimodal1#nodejs1#normalisation1#notifications1#ocr1#offline-testing1#openai1#overlap1#pagination1#parent-child1#pdf-parsing1#performance1#permissions1#persistence1#pgvector1#pipeline-reliability1#portfolio1#primitives1#priority-queue1#proactive-messaging1#product-engineering1#production-readiness1#prompt1#prompt-assembly1#prompt-bloat1#prompt-design1#prompt-limits1#prompt-surface1#prompt-template1#prompt-versioning1#prompting1#protocol-versions1#quality1#quality-check1#quantization1#query-transformation1#quiet-hours1#rag-basics1#rank-fusion1#reasoning1#reconnect1#redis1#reflection1#replay1#reporting1#retrieval-failure1#retrieval-metrics1#retry-semantics1#retry-strategy1#review1#risk-assessment1#rollback1#rollout1#rrf1#safety1#sampling1#sandbox1#sandboxing1#scaling1#schema-design1#schema-validation1#scoping1#secrets-management1#self-assessment1#self-introduction1#self-presentation1#self-reflection1#server-design1#service-architecture1#session-management1#sessions1#similarity1#skill-authoring1#skill-description1#skill-design1#spec1#split-brain1#stakeholder-communication1#star1#state-persistence1#stateless1#stdio-transport1#stopping-criteria1#storytelling1#subagent1#subscriptions1#subtitles1#task-graph1#team-governance1#test-set1#test-strategy1#testing1#thresholds1#timezone1#token-accounting1#token-budget1#tool-budget1#tool-execution1#tool-naming1#tool-schema1#trust-boundary1#tuning1#validation1#vector-index1#workflow-design1#xml-tags1#zero-downtime1
30 天从前端工程师到 Agent 工程师
D5 工具系统与事件驱动:参数校验、错误回传让模型自纠错、事件订阅(dg P05/P06/M05/M07)
设计 Agent 的工具时你会遵循哪些原则?名字、描述、参数分别该怎么写?What principles do you follow when designing tools for an agent — how do you write the name, the description and the parameter schema?
国内高频海外高频基础#tool-design#prompt-engineering分析过程 · 先想清楚再作答
- 这题的区分度在于你把工具描述当成什么。当成函数注释的人会答「写清楚做什么」,当成提示词的人才会答到点子上——描述会原样进入模型的上下文,参与「该不该调、参数填什么」的判断,它的读者是模型不是同事。
- 拆成三件事分别说:名字要动词加宾语(query_order 而不是 handler2),因为名字是模型的第一道筛选;描述最有价值的一句不是「做什么」而是「什么时候不该用它」,把边界写进去能砍掉一大半误用;参数里每个字段都要有自己的 description,格式类字段还要给一个合法示例——模型对「订单号」没有概念,看到 SO20260901 这个样例,填对的概率会陡增。
- 接着给出一条几乎没人主动说的成本判断:工具定义每一轮都会被完整重发,一个写得扎实的工具约 100 到 150 token,挂 20 个就是每轮两三千 token 的固定开销。所以「工具越多越强」是错的,只挂当前场景用得上的那几个。
- 再补一条可迁移的工程判断:工具改名或改语义是破坏性变更,等价于换了个工具——调好的提示词会失效,历史会话的 messages 里还留着旧名字,恢复旧会话时模型会去调一个不存在的工具。所以改工具要像改公开 API 一样走版本与灰度。
- 可以预期的追问:几十上百个工具怎么办?答案是先用一轮便宜模型做工具检索,只把最相关的几个塞进正式请求,而不是一股脑全挂上。
How to reason about it · think before answering
- The discriminator is what you think a tool description is. People who treat it as a docstring answer 'describe what it does'; people who treat it as part of the prompt get it right — the description goes verbatim into the model's context and drives both tool selection and argument filling. Its reader is the model, not your teammate.
- Split it into three: names read like commands (query_order, not handler2) because the name is the model's first filter; the most valuable sentence in a description is not what the tool does but when NOT to use it, which removes most misrouting; and every parameter needs its own description plus a concrete example for format-shaped fields — a model has no notion of 'order id', but SO20260901 makes it far more likely to get it right.
- Then raise the cost point most candidates miss: tool definitions are resent in full every turn. A well-written tool runs 100 to 150 tokens, so twenty of them is a fixed two- to three-thousand-token tax per turn. More tools is not more capable — only mount what the current scenario needs.
- Add a transferable engineering judgment: renaming a tool or silently widening its semantics is a breaking change. Tuned prompts stop working, and old sessions still carry the old name in messages, so resuming one makes the model call a tool that no longer exists. Version and roll out tool changes the way you would a public API.
- Expect the follow-up: what about dozens or hundreds of tools? Retrieve tools with a cheap model first and mount only the top few, rather than shipping the whole catalog every turn.
答题要点
- 工具定义是提示词的一部分,读者是模型:它只能看到名字、描述、参数 schema,看不到你的实现
- 名字用动词加宾语;描述里最值钱的是「什么时候不该用它」;每个参数都要有 description,格式类字段给一个合法示例
- 工具定义每轮完整重发,20 个工具就是每轮固定两千多 token,只挂当前场景用得上的
- 改名或改语义等于换工具,会让调好的提示词失效、让旧会话调到不存在的工具,要走版本与灰度
- 工具规模上去之后,先用便宜模型做工具检索再挂载最相关的几个
Key points
- A tool definition is part of the prompt; the model only sees name, description and parameter schema
- Name it verb plus object; the most valuable line in a description is when not to use it; every parameter needs a description, and format fields need a concrete example
- Definitions are resent every turn, so twenty tools is a fixed two- to three-thousand-token tax — mount only what the scenario needs
- Renaming or redefining a tool is a breaking change that invalidates tuned prompts and breaks resumed sessions
- At scale, retrieve the relevant tools with a cheap model before mounting them
D12 长期记忆:pgvector、embedding、chunking、memory_search 工具
把记忆检索包装成 memory_search 这样的工具时,参数设计上要注意什么?What matters when designing the parameters of a retrieval tool such as memory_search?
国内高频海外高频深入#tool-design#security#long-term-memory分析过程 · 先想清楚再作答
- 这题看着是接口设计题,真正的区分度在安全。多数人会答 query 和 limit,答完就停;能不能说出「哪些参数绝对不能给模型」,决定了这题的分数。
- 先说 query:描述里要写清它不是用户原话,而是模型自己组织的检索语句,并给一个合法示例。用户问「我有什么忌口」,模型应该用「用户的食物忌口」去检索——这是从 D5 那条「格式类字段要给合法示例」延续下来的。
- 再说 limit:可选、默认 5、上限 10。上限的理由不是防呆,是上下文预算——一条记忆约 400 token,10 条就是 4000 token 进请求。模型传 50 时按工具协议回一条可读错误让它改,而不是静默截断成 10,否则模型永远不知道自己传错了。
- 然后是关键的一条:**绝不给 user_id 这类身份参数**。用户身份只能来自会话上下文。做成参数等于把「查谁的记忆」交给一段概率生成的文本,配上一句提示词注入就是现成的越权读取漏洞。一句话收尾:提示词管意图,代码管权限。
- 返回格式同样要说:空结果必须显式返回一句「没有找到相关记忆,请不要凭空推测」,返回空串模型会当成没有约束然后自己编;把相似度分数一起返回,模型才能区分「你说过」和「我印象里你好像提过」;设一条相似度下限,宁可不返回也不要拿噪声污染上下文。
- 可以预期的追问:检索不到的时候该怎么办?答案是分两层——工具层如实返回空并禁止推测,提示词层要求模型转而向用户确认,而不是把「没检索到」当成「用户没有偏好」。
How to reason about it · think before answering
- It looks like an API design question; the discriminating part is security. Most candidates name query and limit and stop. Saying which parameters must never be exposed to the model is what earns the point.
- On query: the description must state that it is a retrieval phrase the model composes, not the user's literal words, and give a concrete example. Asked 'what are my dietary restrictions', the model should search for 'the user's food allergies and restrictions'.
- On limit: optional, default 5, capped at 10. The cap is a context budget, not idiot-proofing — a memory is roughly 400 tokens, so ten of them put 4000 tokens into the request. When the model asks for 50, return a readable validation error naming the field, the valid range and an example, rather than silently clamping, or it never learns it was wrong.
- The critical rule: never expose an identity parameter such as user_id. Identity comes from the session. Making it a parameter hands 'whose memories to read' to probabilistically generated text, and one prompt injection turns it into a privilege-escalation read. Prompts govern intent; code governs permission.
- Cover the response shape too: an empty result must say so explicitly and forbid guessing, because an empty string reads to the model as 'no constraints' and invites fabrication; return similarity scores so the model can distinguish a firm memory from a vague one; and set a minimum score, since no result beats a noisy one.
- Expect: what should happen on a miss? Two layers — the tool returns empty honestly and forbids speculation, and the prompt instructs the model to ask the user instead of treating 'not found' as 'no preference'.
答题要点
- query 必填,描述里说明它是模型组织的检索语句而非用户原话,并给一个合法示例
- limit 可选、默认 5、上限 10,上限的依据是上下文预算;超限按工具协议回可读错误让模型改,不要静默截断
- 绝不把 user_id 这类身份参数交给模型,身份只能来自会话——否则一句提示词注入就是越权读取
- 空结果要显式说「没找到,请不要凭空推测」,返回空串模型会自己编
- 返回相似度分数并设下限,宁可不返回也不要用低相关记忆污染上下文
Key points
- query is required; document it as a model-composed retrieval phrase, not the user's literal words, with an example
- limit is optional, defaults to 5 and caps at 10 on context-budget grounds; over the cap, return a readable validation error instead of silently clamping
- Never expose user_id or any identity parameter — identity comes from the session, or prompt injection becomes a privilege-escalation read
- An empty result must say so explicitly and forbid speculation, or the model fabricates
- Return similarity scores and enforce a minimum, since no result beats a noisy one
7 天 MCP:把工具接进任何 Agent
D2 写第一个 MCP server:stdio 传输、官方 SDK、参数 schema、工具注解与 Inspector 调试
工具的 description 到底写给谁看?写得太泛,在生产里会造成什么具体后果?Who is a tool's description actually written for, and what concretely goes wrong in production when it is too vague?
国内高频海外高频基础#tool-design#prompt-surface分析过程 · 先想清楚再作答
- 这题在筛「有没有真的排查过模型不调工具」。答成「写清楚一点,方便别人理解」就落到文档思维了;面试官想听的是描述是模型唯一的判断依据这件事。
- 拆法:先问自己「模型做这个决定时手上有什么」。它看不到你的 wiki、代码注释、需求文档,只有工具名加这一句描述加参数 schema。所以描述不是文档,是决策依据。
- 把「太泛」拆成两个方向的后果:一是**漏调**,模型不知道这个工具能解决当前问题,任务默默做不成,而且不会报错;二是**误调**,描述边界不清,模型在不该调的时候调它——如果这个工具有副作用,那就是一次真实的线上事故。
- 结论:一句合格的描述要回答三件事——做什么、参数长什么样(给例子)、什么情况下才该用。第三条最常被漏掉,也最要命,因为它才是防误调的那道闸。
- 补一条工程视角:描述是对外契约,改它等于改行为。同一段描述在不同模型上表现还不一样,所以描述要进版本管理、要有评估集,不能靠上线后人肉观察。
- 可预期的追问:那把描述写得越长越好吗?不是。描述会占上下文预算,工具一多就挤掉真正的对话内容;正确做法是短而准,把细节放进每个参数各自的 description 里。
How to reason about it · think before answering
- The screen is whether you have ever debugged a tool the model refuses to call. Answering 'write it clearly so colleagues understand' reveals doc-thinking; the point is that the description is the model's only evidence.
- Ask what the model has when it makes the decision: the tool name, this one description, and the parameter schema. It cannot see your wiki, comments, or spec. The description is a decision input, not documentation.
- Split vagueness into two failure directions. Under-calling: the model never realizes the tool solves the current problem, so the task silently fails with no error. Over-calling: fuzzy boundaries make the model invoke it when it should not, which is a real incident if the tool has side effects.
- Conclusion: a usable description answers three things — what it does, what the parameters look like with an example, and when it should be used. The third is the one people omit, and it is the gate that prevents over-calling.
- Add the engineering view: a description is an external contract, so changing it changes behavior, and the same wording performs differently across models. It belongs in version control with an eval set, not in post-launch eyeballing.
- Likely follow-up: is longer always better? No. Descriptions consume context budget and crowd out the actual conversation once you have many tools. Keep the summary short and push detail into each parameter's own description.
答题要点
- 描述是给模型看的,是它决定调不调这个工具的唯一依据,不是给同事看的文档
- 写得太泛有两类后果:漏调导致任务静默失败,误调则可能触发有副作用的操作
- 合格描述回答三件事:做什么、参数长什么样并给例子、什么情况下才该用
- 描述是对外契约,要进版本管理并配评估集;细节放进每个参数的 description,总描述保持短而准
Key points
- The description is read by the model and is its only basis for deciding whether to call the tool
- Vagueness causes silent under-calling or dangerous over-calling of side-effecting tools
- A good description states what it does, what the parameters look like with an example, and when it applies
- Treat it as an external contract with version control and evals; push detail into per-parameter descriptions to save context
什么时候该返回 JSON-RPC 的 error,什么时候该返回 isError 为真的工具结果?给我一个判据。When should a tool return a JSON-RPC error versus a result with isError set to true? Give me a decision rule.
国内高频海外高频进阶#error-handling#tool-design分析过程 · 先想清楚再作答
- 这题几乎是 MCP 服务端的入门分水岭。能背出「两类错误」只算及格,区分度在于能不能给出一条可执行的判据,以及知不知道错误文案是写给谁的。
- 拆法:问「谁能修好这个错」。请求本身不合法——工具名不存在、参数不满足调用工具的 schema、服务端内部异常——模型再怎么改参数都没用,这类走 JSON-RPC 的 error,典型是 -32602。工具跑了但业务没成——下游 API 失败、日期格式不对、金额越界——模型换个参数就可能成功,这类走 result 里的 isError。
- 判据一句话:**模型换个参数有没有可能成功?有就用 isError,没有就用 error。** 注意 isError 仍然是一个成功的 JSON-RPC 响应,resultType 照样是 complete。
- 结论要带上文案要求:规范说客户端应当把执行错误交给模型自我纠正,所以文案是写给模型看的,要列出可选值、正确格式、边界条件。写「参数错误」等于让模型瞎猜。
- 生产视角的坑:最危险的不是分错类,而是**两类都不返回**——不做校验,让非法输入算出 NaN 或空结果静默返回。模型会把错误答案当正确答案用下去,且不留痕迹。靠输出 schema 校验去兜底也不算处理,因为模型拿到的是一段 schema 堆栈。
- 可预期的追问:客户端要不要把协议错误也喂给模型?规范说可以,但基本没用,因为模型改不了;更该做的是记日志报警,那是你的 bug 不是模型的。
How to reason about it · think before answering
- This is close to a pass/fail line for MCP server work. Reciting 'two kinds of errors' is baseline; the discriminator is producing an actionable rule and knowing who the error text is written for.
- Ask who can fix it. If the request itself is invalid — unknown tool, arguments failing the call-tool schema, an internal server fault — no amount of parameter tweaking helps, so return a JSON-RPC error, typically -32602. If the tool ran but the business case failed — downstream API error, bad date format, amount out of range — a different argument might work, so return isError in the result.
- The rule in one line: could the model succeed by changing an argument? If yes use isError, if no use error. Note that isError is still a successful JSON-RPC response with resultType complete.
- Carry the text requirement into the conclusion: the spec says clients should hand execution errors to the model for self-correction, so the message is written for the model. List allowed values, the correct format, the boundary. 'Invalid parameter' just makes it guess.
- The production trap is not misclassifying but returning neither — skipping validation so an illegal input yields NaN or an empty result that is silently returned. The model then uses a wrong answer with no trace. Leaning on output-schema validation is not handling it either, since the model receives a schema stack trace.
- Likely follow-up: should clients feed protocol errors to the model too? The spec permits it but it rarely helps, because the model cannot fix them. Log and alert instead — that one is your bug.
答题要点
- 协议错误走 JSON-RPC 的 error:未知工具、请求不满足 schema、服务端内部错,模型改参数也无济于事
- 执行错误走结果里的 isError 为真:下游失败、业务校验不过,它仍是成功的 JSON-RPC 响应
- 判据是模型换个参数有没有可能成功,有就 isError,没有就 error
- 执行错误的文案写给模型看,要列出可选值与正确格式;最危险的是两类都不返回、静默给出错误结果
Key points
- Protocol errors use the JSON-RPC error field: unknown tool, schema-invalid request, internal fault — unfixable by the model
- Execution errors use isError true in the result and remain a successful JSON-RPC response
- The rule: if a different argument could succeed, use isError; otherwise use error
- Write execution-error text for the model with allowed values and formats; the worst case is neither, silently returning a wrong result
5 天上下文工程
D3 工具结果与检索的上下文管理:按需加载、摘要与裁剪、结构化返回
Agent 挂了三十个工具,明显吃不消了。你会怎么裁?依据是什么?An agent has thirty tools mounted and it is clearly struggling. How do you cut the list, and on what basis?
国内高频海外高频进阶#tool-design#tool-budget分析过程 · 先想清楚再作答
- 这题在考你能不能区分两类完全不同的代价。只答「工具定义占 token」的人只看到了一半,面试官真正在意的是另一半——选择成本。
- 怎么拆:先分两类问题。预算问题是工具定义每一轮都重发,3 个工具约 308 token、8 个约 731,挂三十个就是两千多,每轮都在花。选择问题是工具越多模型越容易选错,判据很硬:如果一个人类工程师都说不清什么时候该用哪个工具,那模型也做不到。
- 给四条可执行的裁剪判据,按顺序问:职责有没有重叠(有就合并或把边界写进描述)、过去一百次真实会话里被调用过几次(零次直接摘掉)、任务类型能不能提前判断(能就按任务动态挂载)、能不能把几个查询合并成一个带参数的工具。
- 结论:第三条通常收益最大。多数 Agent 的工具清单是静态的,启动时挂全集挂到会话结束,而一次会话往往只属于两三类任务中的一类,按任务大类动态挂载一步就能砍掉一半。
- 可预期的追问:动态挂载有什么副作用?工具定义排在缓存前缀最前面,改它会让整段前缀失效,所以只能按任务大类切几档,不能每轮重算——会话开始时定一次,中途除非任务类型真变了否则不动。
How to reason about it · think before answering
- The question separates people who see only the token bill from people who also see the selection cost. Answering with token count alone covers half the problem.
- Two distinct costs. Budget: tool definitions are resent every turn, roughly 308 tokens for three tools and 731 for eight, so thirty tools burn a couple thousand tokens per turn. Selection: more tools means more wrong choices, and the sharp test is that if a human engineer cannot say which tool applies, the model cannot either.
- Give four ordered criteria: overlapping responsibility (merge, or write the boundary into the description), call frequency across the last hundred real sessions (zero calls means remove), whether the task type can be determined up front (if so, mount per task), and whether several query tools can collapse into one parameterized tool.
- Conclusion: the third usually wins biggest. Most agents mount the full set at startup and keep it for the whole session, while any given session belongs to only two or three task classes, so per-task mounting typically halves the definitions immediately.
- Expect the follow-up on side effects: tool definitions sit at the very front of the cache prefix, so changing them invalidates everything after. Mount by coarse task class once at session start rather than recomputing every turn.
答题要点
- 两类代价:预算(工具定义每轮重发,随数量线性增长)与选择(重叠工具让模型在决策点上摇摆)。
- 判据:人类说不清该用哪个,模型也做不到。
- 四条裁剪顺序:合并职责重叠的、摘掉零调用的、按任务类型动态挂载、把多个查询合并成带参数的一个。
- 动态挂载收益最大,但会打掉缓存前缀,所以按任务大类切档、会话内不再变。
Key points
- Two costs: budget (definitions resent every turn, growing with count) and selection (overlapping tools make the model waver at decision points).
- Test: if a human cannot say which tool applies, neither can the model.
- Four cuts in order: merge overlaps, drop never-called tools, mount per task type, collapse several queries into one parameterized tool.
- Per-task mounting pays most but invalidates the cache prefix, so switch by coarse task class once per session.
14 天 RAG:从检索到可信回答
D12 Agentic RAG:把检索做成工具,让模型自己决定查不查、查几次、要不要推翻重来
把检索包成一个工具交给模型,这个工具的描述该怎么写?写不好会导致哪些具体的错误行为?You are exposing retrieval to a model as a tool. How do you write the tool description, and what concrete failure modes appear when you write it badly?
国内高频海外高频基础#tool-design#agentic-rag#prompting分析过程 · 先想清楚再作答
- 这题的题眼是「具体的错误行为」。只会背「描述要写清楚工具的用途」的,一句话就暴露了没上过线——面试官想听的是描述里少一句话,线上就多一类工单。
- 先给结构:一段合格的工具描述要回答四件事——库里有什么和没有什么、什么时候必须用、什么时候不要用、查询串写成什么形状。四条各对应一类事故,逐条挂钩着说最有说服力。
- 逐条挂钩:不写范围,模型拿它当搜索引擎,问天气也去查;不写「必须用」,涉及公司制度的问题被模型凭记忆编答案,而且编得非常像真的;不写「不要用」,闲聊和翻译都触发一次无谓检索,成本和延迟白涨;不写查询形状,模型把用户整句问话塞进 query,「叫什么名字」这种疑问词进了检索,纯噪声。
- 最后一条最值钱也最容易漏:在描述里加一句「写成关键词短语,不要带疑问词」,比在检索侧做十种查询清洗都管用——问题在源头,就在源头修。
- 补一个生产视角:参数里的过滤字段(比如部门)要写明「只在确定时才填」。模型倾向于把可选参数填满,填错一个部门就把正确答案挡在库外,而这种错误在日志里看不出来,表现是「检索没结果」。
- 可预期的追问是「怎么验证描述写对了」。答案是拿一批负样本跑:闲聊、翻译、算术、以及答案已在对话里的追问,看模型有没有多调一次工具;这类回归是能自动化的。
How to reason about it · think before answering
- The discriminator is whether you can name concrete failure modes. Reciting 'the description should be clear' signals you have never shipped one.
- Give the structure first: a usable description answers four things - what is and is not in the corpus, when the tool must be called, when it must not be called, and what shape the query string should take.
- Attach a failure to each: no scope and the model treats it as a web search; no 'must call' and it answers policy questions from memory, convincingly; no 'must not call' and greetings or translations each burn a retrieval; no query shape and the model pastes the raw user sentence in, dragging interrogative words into the index.
- The query-shape line is the cheapest win: one sentence saying 'keyword phrase, no question words' beats ten heuristics for query cleaning on the retrieval side.
- Production angle: optional filter parameters such as department need an explicit 'only set this when you are certain'. Models like to fill optional fields, and a wrong filter hides the correct answer while the logs only show 'no results'.
- Expected follow-up: how do you verify the description works? Run a negative suite - small talk, translation, arithmetic, follow-ups already answered in the conversation - and assert the tool was not called. That regression is automatable.
答题要点
- 描述是写给模型看的提示词,不是注释;四段式:范围、什么时候用、什么时候不用、查询写成什么形状。
- 不写范围会被当成搜索引擎;不写「必须用」会导致凭记忆编答案。
- 不写「不要用」会让闲聊也触发检索,成本和延迟白涨。
- 写明查询要用关键词短语、不带疑问词,比在检索侧清洗查询更根本。
- 可选过滤参数要写「只在确定时才填」,填错会静默地把正确答案挡在外面。
- 用一批负样本(闲聊、翻译、算术)做回归,断言工具没有被调用。
Key points
- The description is a prompt for the model, not a code comment: scope, when to call, when not to call, query shape.
- Missing scope turns it into a web search; missing 'must call' produces confident answers from memory.
- Missing 'do not call' makes small talk trigger retrieval, paying cost and latency for nothing.
- Stating 'keyword phrase, no question words' fixes query pollution at the source.
- Optional filters need 'only set when certain' - a wrong filter silently hides the right answer.
- Regression-test with a negative suite and assert the tool was not invoked.