面试题库
共 328 题,当前筛选 9 题。
课程全部30 天从前端工程师到 Agent 工程师5 天提示词工程零基础Claude 高效使用:从对话到 Claude CodeCodex 与 OpenAI Agents SDK 高效使用7 天 MCP:把工具接进任何 Agent7 天 Agent Skills:把经验做成可复用能力5 天上下文工程14 天 RAG:从检索到可信回答14 天用 Agent 搭一条 AI 短剧生产线
标签
全部#coding-agent9#evaluation30#cost29#reliability24#architecture22#agent-skills21#security18#observability15#streaming12#error-handling11#idempotency10#system-design10
还有 359 个标签收起标签
#api-design9#distributed-systems8#multi-agent8#rag8#chunking7#debugging7#pipeline-design7#structured-output7#agent-loop6#mcp6#operations6#prompt-injection6#sse6#tool-calling6#tool-design6#context5#context-engineering5#deployment5#embeddings5#hybrid-search5#message-bus5#scheduling5#system-prompt5#agentic-rag4#behavioral4#client4#concurrency4#consistency4#framework-design4#ingestion4#prompt-engineering4#rate-limiting4#retrieval4#routing4#trade-offs4#abstention3#agents-sdk3#caching3#communication3#context-window3#cost-control3#data-quality3#failure-modes3#image-generation3#interview-prep3#langgraph3#latency3#llm-as-judge3#llm-basics3#long-context3#model-migration3#model-routing3#orchestration3#ordering3#progressive-disclosure3#prompt-basics3#prompt-caching3#protocol3#provider-abstraction3#ranking3#recall3#redis-streams3#responses-api3#resume3#retry3#runtime3#scalability3#skills3#state-management3#statelessness3#subagents3#timeline3#versioning3#workflow-engine3#access-control2#agent-design2#async-task2#auth2#checkpointing2#citation-verification2#citations2#claude-md2#code-review2#compaction2#context-rot2#contextual-retrieval2#cost-tradeoff2#data-modeling2#database2#distribution2#fallback2#ffmpeg2#golden-set2#grounding2#interview-process2#long-term-memory2#media-pipeline2#memory2#multi-hop2#multi-tenancy2#oauth2#prioritization2#project-storytelling2#prompt-techniques2#query-rewriting2#react2#refusal2#reproducibility2#rerank2#retrieval-quality2#retrospective2#scripts2#sharding2#state-machine2#tool-permissions2#tooling2#tools2#tracing2#transport2#tts2#ux2#vector-database2#verification2#workflow2#agent-basics1#agent-quality1#agent-sdk1#agents-md1#altitude1#analytics1#approvals1#architecture-review1#async1#atomicity1#attention-budget1#av-sync1#backoff1#bi-encoder1#bm251#budget-control1#build-vs-buy1#cancellation1#candidate-selection1#capacity-planning1#career1#chain-of-thought1#ci1#circuit-breaker1#claude-code1#cli-design1#client-integration1#cloud1#compression1#configuration1#confused-deputy1#consistent-hashing1#content-hash1#content-safety1#context-assembly1#context-compression1#context-management1#coreference1#correctness1#cost-accounting1#cost-analysis1#cost-optimization1#cross-encoder1#cross-model1#customer-support1#dag1#decision-making1#decomposition1#degradation1#deliberate-practice1#design1#diagnostics1#dimensions1#docker1#documentation1#embedding-migration1#encoding1#engineering-judgement1#engineering-tradeoffs1#error-propagation1#escalation1#eval1#event-driven1#evidence1#failure-analysis1#fairness1#faithfulness1#fan-out1#feedback-loop1#fencing-token1#few-shot1#filter-pushdown1#filtering1#fine-tuning1#forking1#four-elements1#framework-selection1#frontend1#global-market1#graph-rag1#guardrails1#hallucination1#handoff1#handoffs1#headless1#hnsw1#hooks1#human-in-the-loop1#hybrid1#hyde1#incremental-recompute1#incremental-sync1#index-maintenance1#index-routing1#indexing1#information-retrieval1#instruction-hierarchy1#integration1#intent-routing1#interrupt-merge1#invalidation1#isolation1#iterative-scan1#ivfflat1#json-parsing1#json-schema1#just-in-time1#jwt1#knowledge-organization1#latency-budget1#lease1#least-privilege1#llm-output-quality1#long-session1#loop-guard1#maintenance1#mcp-basics1#mental-model1#messages-api1#metadata1#methodology1#metrics1#migration1#mobile1#model-selection1#moderation1#modularity1#multi-turn1#multimodal1#nodejs1#normalisation1#notifications1#ocr1#offline-testing1#openai1#overlap1#pagination1#parent-child1#pdf-parsing1#performance1#permissions1#persistence1#pgvector1#pipeline-reliability1#portfolio1#primitives1#priority-queue1#proactive-messaging1#product-engineering1#production-readiness1#prompt1#prompt-assembly1#prompt-bloat1#prompt-design1#prompt-limits1#prompt-surface1#prompt-template1#prompt-versioning1#prompting1#protocol-versions1#quality1#quality-check1#quantization1#query-transformation1#quiet-hours1#rag-basics1#rank-fusion1#reasoning1#reconnect1#redis1#reflection1#replay1#reporting1#retrieval-failure1#retrieval-metrics1#retry-semantics1#retry-strategy1#review1#risk-assessment1#rollback1#rollout1#rrf1#safety1#sampling1#sandbox1#sandboxing1#scaling1#schema-design1#schema-validation1#scoping1#secrets-management1#self-assessment1#self-introduction1#self-presentation1#self-reflection1#server-design1#service-architecture1#session-management1#sessions1#similarity1#skill-authoring1#skill-description1#skill-design1#spec1#split-brain1#stakeholder-communication1#star1#state-persistence1#stateless1#stdio-transport1#stopping-criteria1#storytelling1#subagent1#subscriptions1#subtitles1#task-graph1#team-governance1#test-set1#test-strategy1#testing1#thresholds1#timezone1#token-accounting1#token-budget1#tool-budget1#tool-execution1#tool-naming1#tool-schema1#trust-boundary1#tuning1#validation1#vector-index1#workflow-design1#xml-tags1#zero-downtime1
Codex 与 OpenAI Agents SDK 高效使用
D1 Codex CLI 入门:安装、AGENTS.md、审批模式与沙箱、常用命令
给 coding agent 写的项目说明文件(比如 Codex 的 AGENTS.md)应该写什么、不该写什么?为什么它要有大小上限?What belongs in a project instruction file for a coding agent (such as Codex's AGENTS.md), what does not, and why is there a size limit?
国内高频海外高频基础#coding-agent#context#agents-md分析过程 · 先想清楚再作答
- 这题考的不是文件格式,而是你对「上下文是有限资源」有没有工程直觉。把它答成「写项目介绍」会被判为没真用过。
- 拆法是一个判断句:这条信息 agent 打开文件自己能不能发现?能发现的不写(目录结构、用了什么框架),发现不了的才写(约定、禁区、环境事实、测试命令)。
- 再补一层查找规则:全局层在用户目录,项目层从根目录到当前目录依次拼接,越靠近当前目录越靠后、越优先,所以子目录可以覆盖根规则。
- 大小上限(Codex 默认 32 KiB)的意义是逼你做取舍:手册太长会挤占任务本身的上下文,还会让模型对每一条规则的遵守度下降。
- 可预期的追问:写在说明文件里的规则模型一定会遵守吗?不一定,它是提示词的一部分,会被长对话稀释;硬约束要靠沙箱与审批,不是靠文字。
How to reason about it · think before answering
- This probes whether you treat context as a scarce resource, not whether you know the file format; answering with a project overview signals inexperience.
- Use one test: can the agent discover this by opening files? If yes, leave it out (directory layout, framework); if no, write it down (conventions, no-go areas, environment facts, test commands).
- Add the lookup rules: a global file in the home directory, then project files concatenated from the repo root down to the current directory, so closer files override earlier ones.
- The size cap (32 KiB by default in Codex) forces prioritization: a long manual crowds out the task and dilutes adherence to every rule.
- Expect the follow-up: will the model always obey the file? No, it is prompt text and fades over long sessions; hard limits belong to the sandbox and approvals.
答题要点
- 写约定、禁区、环境事实和验证命令;不写 agent 自己打开文件就能发现的内容
- 查找顺序是全局文件在前、项目文件从根到当前目录拼接,越靠近当前目录越优先
- 大小上限逼你只保留高价值信息,避免挤占任务上下文、降低规则遵守度
- 文字规则是建议性的,真正不能越的线交给沙箱与审批
Key points
- Write conventions, no-go areas, environment facts and verification commands; skip anything discoverable from the files
- Lookup goes global first, then project files concatenated root-down, with closer files taking precedence
- The size cap forces you to keep only high-value guidance so the task itself keeps its context budget
- Instruction files are advisory; hard limits come from the sandbox and approval policy
Codex 把「什么时候问用户」和「能碰到什么」拆成 approval_policy 和 sandbox_mode 两组独立开关。为什么要拆?各自解决什么问题?Codex splits 'when to ask the user' and 'what can be touched' into two independent settings, approval_policy and sandbox_mode. Why separate them, and what does each solve?
国内高频海外高频进阶#coding-agent#security#sandbox分析过程 · 先想清楚再作答
- 题眼是「为什么拆」。只背出每组的取值等于没答,面试官要听的是两者正交带来的好处。
- 先给定义:审批策略是流程控制,决定动作执行前要不要人点头;沙箱是权限控制,决定即使模型想做、操作系统允不允许。
- 再说为什么正交:你可能想要「不打扰我,但绝不许出工作区」(on-request 加 workspace-write),也可能想要「每步都问,但只让它读」(untrusted 加 read-only);合成一个滑杆就表达不了这两种组合。
- 落到实现:沙箱靠操作系统机制(macOS Seatbelt、Linux bubblewrap),不是靠模型自觉,所以它是硬约束;审批则是唯一由人把关的环节。
- 可预期的追问:为什么网络默认关?因为联网是把内部代码送出去或把外部代码拉进来的通道,风险等级和改本地文件不同,需要单独授权。
How to reason about it · think before answering
- The discriminating part is 'why separate'; reciting the values without explaining orthogonality earns little.
- Define both: approval policy is process control, whether a human must nod before an action; sandbox is permission control, whether the OS allows the action at all.
- Then justify orthogonality with combinations a single slider cannot express: 'do not interrupt me but never leave the workspace' versus 'ask every time but read-only'.
- Ground it in implementation: the sandbox uses OS mechanisms (Seatbelt on macOS, bubblewrap on Linux) rather than model goodwill, so it is a hard limit, while approval is the one human checkpoint.
- Expect the follow-up: why is network off by default? Because network is the channel for code leaving or entering the machine, a different risk class from local edits.
答题要点
- approval_policy 管流程:untrusted / on-request / on-failure / never 决定动作前是否要人确认
- sandbox_mode 管权限:read-only / workspace-write / danger-full-access 决定操作系统放行什么
- 两者正交才能表达「不打扰但不越界」和「步步问但只读」这类组合
- 沙箱是操作系统级硬约束,审批是唯一的人工把关点;网络默认关闭需单独放开
Key points
- approval_policy governs process: untrusted / on-request / on-failure / never decide whether a human confirms first
- sandbox_mode governs permission: read-only / workspace-write / danger-full-access decide what the OS allows
- Orthogonality lets you express 'no interruptions but stay in the workspace' and 'ask each step but read-only'
- The sandbox is an OS-level hard limit, approval is the human checkpoint, and network is off by default
你要在团队里引入一个能在本地执行命令的 coding agent,怎么向不放心的同事解释它的风险边界?You want to introduce a coding agent that runs commands locally. How do you explain its risk boundary to skeptical teammates?
国内高频海外高频进阶#coding-agent#security#communication分析过程 · 先想清楚再作答
- 这题考的是沟通加工程两层:既要说清技术上的边界,又要用对方能验证的方式说,不能只说「它很安全」。
- 拆成三层防线来讲:第一层文字规则(AGENTS.md)管习惯;第二层沙箱管能力,只读或只能写工作区、网络默认关;第三层审批管例外,越界的每一步都要人批。
- 给出可验证的承诺:所有改动都在 git 工作区里,`git diff` 能看、`git checkout` 能撤;脱手运行只跑在一次性分支或容器里。
- 主动说出剩余风险:模型可能误读需求写出错误但能通过的代码,所以审查和测试不能省;密钥不要放在它能读到的文件里。
- 可预期的追问:能不能完全禁止它联网?可以,沙箱默认就不通网,需要装依赖时逐次批准,或在配置里给一个允许的域名清单。
How to reason about it · think before answering
- This tests communication as much as engineering: state the technical boundary in terms the listener can verify, not just 'it is safe'.
- Present three layers of defense: written rules (AGENTS.md) shape habits; the sandbox limits capability to read-only or workspace-only writes with network off; approvals gate every exception.
- Offer verifiable guarantees: every change lands in the git working tree, visible via diff and revertable via checkout; unattended runs stay on throwaway branches or containers.
- Name the residual risk yourself: the model can misread a requirement and produce wrong but passing code, so review and tests remain mandatory, and secrets stay out of readable files.
- Expect the follow-up: can network be fully blocked? Yes, the sandbox is offline by default; approve installs case by case or configure an allow-list of domains.
答题要点
- 三层防线:文字规则管习惯、沙箱管能力、审批管例外
- 改动全在 git 工作区,可 diff 可撤销;脱手运行只在一次性分支或容器
- 主动说明剩余风险:错误但能通过的代码、密钥暴露,所以审查与测试不能省
- 网络默认关闭,联网按次批准或配置允许域名清单
Key points
- Three layers: written rules for habits, the sandbox for capability, approvals for exceptions
- All edits live in the git working tree and are diffable and revertable; unattended runs use throwaway branches or containers
- State residual risks yourself: wrong-but-passing code and secret exposure, hence mandatory review and tests
- Network is off by default; approve per request or configure an allow-list
D2 Codex 进阶:云端任务、代码审查、MCP 接入、自定义指令、IDE 集成
云端 coding agent 能同时跑很多任务,但没有人在旁边点头。它的审批边界应该画在哪里?A cloud coding agent can run many tasks in parallel with nobody around to approve steps. Where should its approval boundary sit?
国内高频海外高频进阶#coding-agent#cloud#approvals分析过程 · 先想清楚再作答
- 这题考的是你有没有意识到「审批模型变了」:本地是逐步审批,云端只能事先授权、事后审阅。答成「跟本地一样弹窗」说明没用过。
- 拆法是把边界分成三个时间点:任务开始前(环境配置决定能联网什么、有哪些变量、装什么依赖)、任务执行中(容器隔离,改动只在容器里)、任务结束后(人审 diff 再决定开不开 PR)。
- 结论是:云端的审批边界就是「环境配置 + PR 前人工审阅」这两道门,中间不再有人;所以生产密钥不能进环境、公网默认关、合并权限保留在人手里。
- 补一条工程视角:并行任务之间的边界也要画——互相会改同一批文件的任务不要同时派,否则合并成本吃掉并行收益。
- 可预期的追问:能不能让它自动合并?可以在低风险仓库对通过全部测试的 PR 这么做,但要保留回滚手段,并且把「自动合并」本身当成一个需要审批的配置变更。
How to reason about it · think before answering
- This checks whether you noticed the approval model changed: local means step-by-step approval, cloud means authorize upfront and review afterwards.
- Split the boundary across three moments: before the task (environment config decides network, variables, dependencies), during (container isolation), after (a human reviews the diff before any PR).
- Conclude that the cloud boundary is two gates, environment config plus pre-PR human review, with nobody in between; hence no production secrets, network off by default, merge rights stay human.
- Add the engineering angle: draw boundaries between parallel tasks too; tasks that touch the same files should not run concurrently.
- Expect the follow-up: can it auto-merge? Only in low-risk repos for fully green PRs, with rollback in place, and treat enabling auto-merge as a change that itself needs approval.
答题要点
- 云端没有逐步审批,边界变成事前的环境配置与事后的人工审阅两道门
- 环境里不放生产密钥、公网默认关、合并权限保留给人
- 并行任务之间也要画边界:会改同一批文件的任务不同时派
- 自动合并只适用于低风险仓库且全绿的 PR,并保留回滚
Key points
- No step-wise approval in the cloud; the boundary becomes upfront environment config plus post-hoc human review
- Keep production secrets out, network off by default, merge rights with humans
- Draw boundaries between parallel tasks: never run file-overlapping tasks concurrently
- Auto-merge only for low-risk repos with fully green PRs, with rollback ready
让同一个模型既写代码又审代码,审查还有意义吗?怎么让审查更独立?If the same model both writes and reviews code, is the review still meaningful? How do you make it more independent?
国内高频海外高频进阶#code-review#coding-agent#workflow分析过程 · 先想清楚再作答
- 题眼在「还有意义吗」——直接答「没意义」或「有意义」都不及格,要说清它能抓什么、抓不到什么。
- 先说能抓的:审查时输入变了(看 diff 而不是需求)、立场变了(找问题而不是完成任务),这种角色切换能抓出漏掉的边界情况、没同步的调用方、明显的风格违规。
- 再说抓不到的:审查者和生成者共享同一份对需求的理解,需求理解错了两边一起错;也共享同样的盲区与偏好。
- 结论给三条提高独立性的手段:换一家模型审、给审查者不同的信息(需求原文加验收标准而不是只给 diff)、用确定性工具(测试、lint、类型检查)做第一道审查。
- 可预期的追问:审查意见要不要自动应用?不要,审查是输入不是判决,误报与漏报都存在,最终判断留给人。
How to reason about it · think before answering
- The crux is 'still meaningful'; a flat yes or no fails. Explain what it catches and what it misses.
- What it catches: the input changes (diff instead of requirements) and the stance changes (find faults instead of finish the job), which surfaces missed edge cases, unsynced callers and style violations.
- What it misses: reviewer and author share one understanding of the requirement, so a misread requirement passes; they share blind spots too.
- Conclude with three independence levers: review with a different vendor's model, feed the reviewer different information (original requirement plus acceptance criteria, not just the diff), and run deterministic checks first.
- Expect the follow-up: auto-apply review comments? No; review is input, not verdict, and both false positives and misses exist.
答题要点
- 有意义:输入与立场的切换能抓出边界情况、未同步的调用方、风格违规
- 抓不到与需求理解相关的错误,因为审查者与生成者共享同一份理解
- 提高独立性:换一家模型审、给审查者需求原文与验收标准、先跑确定性检查
- 审查意见是输入不是判决,不要自动应用
Key points
- Yes: the switch of input and stance catches edge cases, unsynced callers and style issues
- It misses requirement misreads because author and reviewer share one understanding
- Increase independence: a different vendor's model, richer reviewer context, deterministic checks first
- Treat comments as input, never auto-apply
MCP server 和 skill 都是在给 coding agent 加能力,什么时候该用哪一个?项目说明文件又放什么?MCP servers and skills both extend a coding agent. When do you reach for each, and what goes in the project instruction file instead?
国内高频海外高频基础#mcp#skills#coding-agent分析过程 · 先想清楚再作答
- 这题考的是抽象层次的区分,是国内面试开始高频出现的「Function Call / MCP / Skills 三者区别」的工具侧版本。
- 拆法是问「加的是什么」:加的是访问外部系统的能力(查工单、读数据库、调内部服务)就是 MCP,它是协议层的工具;加的是一套多步骤的做法(发版检查、迁移流程)就是 skill,它是提示词层的流程包;每次会话都要遵守的约定就是项目说明文件。
- 再给触发方式的差别:MCP 工具由模型在需要数据时调用;skill 由用户显式点名或由模型按描述匹配;说明文件每次会话开头无条件读入。
- 结论落到一句话:规矩归说明文件、外部系统归 MCP、流程归 skill;同一件事只放一处,避免三处互相矛盾。
- 可预期的追问:skill 里能不能调 MCP 工具?可以,skill 的步骤里可以要求使用某个工具,两者是正交的层次,不是替代关系。
How to reason about it · think before answering
- This tests separation of abstraction levels, the tooling-side version of the increasingly common 'function calling vs MCP vs skills' question.
- Ask what is being added: access to an external system (tickets, databases, internal services) is MCP, a protocol-level tool; a multi-step procedure (release checklist, migration flow) is a skill, a prompt-level workflow package; conventions to obey every session belong in the instruction file.
- Contrast triggers: MCP tools are invoked by the model when it needs data; skills are invoked explicitly by name or matched by description; instruction files are loaded unconditionally at session start.
- Conclude: rules in the instruction file, external systems via MCP, procedures as skills; keep each fact in one place to avoid contradictions.
- Expect the follow-up: can a skill use MCP tools? Yes; a skill's steps can call for a tool, the layers are orthogonal, not substitutes.
答题要点
- MCP 加的是访问外部系统的工具,由模型按需调用
- skill 加的是多步骤流程,由用户点名或按描述匹配触发
- 项目说明文件放每次会话都要遵守的约定、禁区与环境事实
- 三者正交:规矩、外部系统、流程各放一处,skill 里可以要求用某个 MCP 工具
Key points
- MCP adds tools that reach external systems, invoked by the model on demand
- Skills add multi-step procedures, triggered by name or matched by description
- The instruction file holds conventions, no-go areas and environment facts read every session
- The three are orthogonal: rules, external systems, procedures each live in one place; a skill may call for an MCP tool
D5 Claude 与 Codex 选型与协作:同一任务双工具实测对比、一个写一个审的混用工作流
团队要在两家 coding agent 之间选一个,你怎么给出一套可以向团队解释、也能被验证的对比维度?Your team must pick between two coding agents. How do you propose a comparison that teammates can both understand and verify?
国内高频海外高频进阶#coding-agent#evaluation#decision-making分析过程 · 先想清楚再作答
- 这题考的是方法论而不是结论。上来就说「我觉得 X 好」会被判为没有工程判断;面试官想听的是你怎么让比较可复现。
- 先给维度:交代(写多少需求、准备多少说明文件)、审批(中断几次、为了什么)、验证(是否主动跑测试、红了怎么办)、成本(时间、token、钱)。这四项都能在自己的仓库里量出来。
- 再给可比性的前置条件:同一个起点 commit、同一段需求文字、说明文件同内容、默认权限、都要求跑完测试再汇报;有一项不同,差异就说不清来源。
- 然后是读数的顺序:先查可比性,再看结构性差异(权限模型、说明文件的位置与措辞导致的行为差别),最后才看能力差异,而且能力差异要多次运行取中位数。
- 落到团队沟通:报告里每一行差异都标「来自工作方式还是能力」,工作方式的差异靠配置弥补,能力差异才影响选型。
- 可预期的追问:榜单为什么不够?榜单测标准题,团队干的是有历史包袱的仓库里的改动,且榜单只给一个分数、不给四个维度。
How to reason about it · think before answering
- This tests methodology, not a verdict; leading with 'I prefer X' signals weak engineering judgment. Show how you make the comparison reproducible.
- Give the dimensions: instruction effort (prompt and instruction-file size), approvals (how many interruptions and why), verification (does it run tests unprompted, what happens on red), cost (time, tokens, money). All are measurable in your own repo.
- State the preconditions for comparability: same starting commit, identical requirement text, identical instruction-file content, default permissions, and 'run tests before reporting' on both sides.
- Then the reading order: check comparability, then structural differences (permission model, placement and wording of rules), and only then capability differences, which need several runs and a median.
- For the team: label every differing row as 'workflow' or 'capability'; workflow gaps are closed by configuration, capability gaps drive the choice.
- Expect the follow-up: why not benchmarks? They score standard problems with one number, while teams change legacy repos and care about four dimensions.
答题要点
- 四个可量维度:交代、审批、验证、成本,全部在自己仓库里测
- 可比性前置:同起点、同需求、同说明文件、默认权限、都要求跑测试
- 读数顺序:可比性、结构性差异、能力差异;能力差异要多次运行取中位数
- 每行差异标「工作方式还是能力」,前者靠配置弥补,后者才决定选型
Key points
- Four measurable dimensions: instruction effort, approvals, verification, cost, all measured in your own repo
- Comparability first: same commit, same prompt, same instruction file, default permissions, tests required
- Read in order: comparability, structural differences, then capability, with medians over several runs
- Label each gap as workflow or capability; only capability gaps should drive the decision
「一家写、另一家审」的混用工作流收益在哪?什么情况下不值得?Where does the 'one vendor writes, the other reviews' workflow pay off, and when is it not worth it?
国内高频海外高频进阶#code-review#workflow#coding-agent分析过程 · 先想清楚再作答
- 题眼在「不值得」。只讲收益不讲代价,是没在预算表前坐过的人的答法。
- 先说收益的来源:同一家模型写与审共享同一份对需求的理解,需求理解偏差抓不出来;换一家审,最大的增量正是这类偏差,其次是不同模型的盲区互补。
- 再说怎么做才有收益:审查方必须拿到需求原文与验收标准而不只是 diff,否则退化成 lint;必须要求结构化输出,否则无法统计采纳率;最后一步必须由人裁决。
- 不值得的三种情况:任务小到审查成本高于任务本身;团队只有一家的额度,跨家意味着双份账单且多抓出的问题不值这笔钱;审查意见没人认真看,多一家只是多一层噪音。
- 最值的三种情况:改动影响多个调用方、需求本身有歧义、改动要上生产——抓出一个理解偏差就回本。
- 可预期的追问:能不能自动化?两家都有脱手模式,写与审都能脚本化,但「人裁决」这一步不能省,否则前两步就是浪费。
How to reason about it · think before answering
- The crux is 'not worth it'; listing benefits without costs reads as never having sat in front of a budget.
- Source of value: when one model both writes and reviews, they share one reading of the requirement, so misreads slip through; a second vendor catches exactly those, plus complementary blind spots.
- How to make it pay: the reviewer needs the original requirement and acceptance criteria, not just the diff, or it degrades into lint; demand structured output so acceptance can be measured; a human makes the final call.
- Not worth it when the task is smaller than the review, when only one vendor's quota exists and the extra bill outweighs extra findings, or when nobody reads review comments carefully.
- Most worth it when a change touches many callers, the requirement is ambiguous, or the change ships to production; one caught misread pays for it.
- Expect the follow-up: can it be automated? Both vendors have headless modes so writing and reviewing can be scripted, but the human adjudication step cannot be removed.
答题要点
- 收益来自独立的需求理解:换一家审能抓出同家审查抓不到的理解偏差
- 审查方要拿到需求原文与验收标准、输出结构化意见,最后由人裁决
- 不值得:任务太小、只有一家额度、没人认真看意见
- 最值:影响多个调用方、需求有歧义、要上生产
Key points
- Value comes from an independent reading of the requirement, catching misreads a same-vendor review misses
- The reviewer needs the requirement and acceptance criteria, must output structured findings, and a human adjudicates
- Not worth it for tiny tasks, single-vendor budgets, or teams that do not read reviews
- Most valuable for multi-caller changes, ambiguous requirements and production deploys
怎么评价一个 coding agent 这次任务的输出质量,而不只是看它跑没跑通?How do you judge the quality of a coding agent's output on a task, beyond whether it ran?
国内高频海外高频深入#coding-agent#evaluation#quality分析过程 · 先想清楚再作答
- 这题考的是你有没有把「测试绿了」当终点。答「看测试」是及格线,区分度在测试之外。
- 拆成四层:正确性(测试是否覆盖了需求里的边界,比如 title 超长、done 传字符串)、契约(错误响应形状是否与需求一字不差,还是它自作主张改了)、范围(有没有改不该改的文件、有没有偷偷加依赖或改默认值)、可维护性(校验规则是否抽成常量、测试是否隔离、命名是否与仓库一致)。
- 再说怎么量:正确性看它写的测试之外你再补的反例能不能过;契约与范围看 diff 与需求逐条对照;可维护性交给第二家模型或人做结构化审查。
- 补一条随机性:单次结果不能下结论,同一需求跑三次看方差,方差大本身就是一个质量信号。
- 可预期的追问:它自己说「已完成并通过测试」能信吗?只信你能复现的部分——在你的机器上重跑测试、看 diff,agent 的汇报是线索不是证据。
How to reason about it · think before answering
- This tests whether you treat green tests as the finish line; 'check the tests' is the pass mark, differentiation lies beyond it.
- Four layers: correctness (do the tests cover the requirement's edges such as overly long titles or a string for done), contract (does the error shape match the spec exactly or did it improvise), scope (did it touch forbidden files, add dependencies or change defaults silently), maintainability (constants extracted, tests isolated, naming consistent with the repo).
- How to measure: correctness by adding your own counterexamples beyond its tests; contract and scope by diffing against the requirement line by line; maintainability via a structured review by a second model or a person.
- Add variance: one run proves nothing; run the same requirement three times and treat high variance as a quality signal in itself.
- Expect the follow-up: can you trust its 'done, tests pass'? Only what you can reproduce; rerun tests and read the diff yourself, the agent's report is a lead, not evidence.
答题要点
- 四层:正确性、契约、范围、可维护性,测试绿只是正确性的一部分
- 正确性用自己补的反例验证,契约与范围对照需求逐条看 diff,可维护性做结构化审查
- 同一需求跑多次看方差,方差大本身是质量信号
- agent 的汇报是线索不是证据,只信自己能复现的部分
Key points
- Four layers: correctness, contract, scope, maintainability; green tests cover only part of correctness
- Verify correctness with your own counterexamples, contract and scope by diffing against the spec, maintainability via structured review
- Run the same requirement several times; high variance is itself a quality signal
- The agent's report is a lead, not evidence; trust only what you reproduce