Interview Bank

Prompt Engineering From Scratch in 5 Days

D1 What a Prompt Is, and Isn't: How the Model Reads Instructions; the Four Elements of Role / Task / Format / Constraints

• What exactly is being engineered in prompt engineering, and how does it differ from writing a requirements doc or a design spec?提示词工程到底在工程什么？它和写需求文档、写技术方案有什么本质区别？

  Common in ChinaCommon overseasBasic#prompt-basics#mental-model

  How to reason about it · think before answering

  1. The screen here is whether the candidate knows the model completes text rather than executes commands. 'Clever wording that makes the model obey' signals chat-app experience only.
  2. Start from the reader: a spec is read by people who share project context; a prompt is read by a completer with zero context that never asks a clarifying question, so every implicit default must be spelled out.
  3. Then justify the word engineering: reproducibility, testability, versioning. A prompt should run against a test set, live in the repo, and diff cleanly between versions.
  4. Conclusion: prompt engineering is making implicit context explicit and managing that text like code; phrasing tricks are a small part.
  5. Likely follow-up: how is that different from a brief for an outsourced team? The team pushes back with questions; the model does not, so a prompt must carry its own completion criteria.

  分析过程 · 先想清楚再作答

  1. 这题在筛「有没有理解模型是在补全而不是执行」。答成「用巧妙的措辞让模型听话」会被判为只会用聊天产品；答出「系统性补齐模型缺少的上下文」才算入门。
  2. 拆法：先问自己「读者是谁」。需求文档的读者是有项目背景的人，可以依赖共享默认；提示词的读者是一个没有任何项目背景、也不会停下来提问的补全器，所有默认信息都得显式写出。
  3. 再落到「工程」二字：可复现、可测试、可版本化。提示词写完要能跑测试集、要进仓库、要能对比两版差异——这才是它区别于「写一段话」的地方。
  4. 结论：提示词工程是把隐性上下文显式化、并把这段文本当代码一样管理的工程活动；措辞技巧只是其中很小的一部分。
  5. 可预期的追问：那和写给外包团队的需求说明有什么区别？答案是外包会反问，模型不会，所以提示词对完整性的要求更高，且要在没有反馈回路的前提下自带完成标准。

  Key points

  • The model completes text rather than executing commands; a prompt is context, and specificity narrows the plausible continuations
  • What gets engineered is the missing information: perspective, completion criteria, output shape, boundaries with reasons
  • Unlike a spec, the reader shares no background and never asks back, so completeness and explicit done-criteria matter more
  • Engineering implies testable, versioned, comparable artifacts, not one-off clever phrasing

  答题要点

  • 模型在补全一段文本而不是执行命令，提示词是给它的上下文，写得越具体可能的下文越窄、输出越稳
  • 工程的对象是「模型缺的信息」：视角、完成标准、输出形状、边界与理由，也就是四要素
  • 区别于需求文档：读者没有共享背景、不会反问，所以完整性要求更高、必须自带完成标准
  • 「工程」意味着可测试、可版本化、可对比，而不是一次性的巧妙措辞

D2 Few-Shot, Chain of Thought, Step-by-Step, and Self-Checks; When None of These Work

• Why does few-shot prompting work, what goes wrong when you give too many examples, and how do you decide how many to include?few-shot 为什么有效？示例给多了会出什么问题？你怎么决定给几个？

  Common in ChinaCommon overseasBasic#few-shot#prompt-techniques

  How to reason about it · think before answering

  1. The screen is whether you treat examples as signals for format and boundaries rather than as magic that makes the model smarter. 'More examples, better model' reads as untested.
  2. Mechanism first: the model completes text, and examples show the continuation directly, which is harder to misread than prose describing a format or an edge rule. Examples are the strongest format signal.
  3. Then the cost: each example consumes context and money; too many cause overfitting to surface features such as length, wording and order, and amplify accidental bias — three bug examples out of four nudges everything toward bug.
  4. Conclusion: the count follows the number of distinct cases you need to cover, typically two to five, each a different case, with at least one boundary sample.
  5. Follow-ups: does order matter? Yes, models weight the last example more, so place the one closest to the target input last. And if examples contradict the instructions, the model usually follows the examples, so they must match the format spec exactly.

  分析过程 · 先想清楚再作答

  1. 这题在筛「有没有把示例当成格式与边界的信号，而不是当成让模型变聪明的魔法」。答成「示例越多模型越懂」会暴露没在生产里调过提示词。
  2. 拆法：先答原理——模型在补全，示例直接展示了「下文该长什么样」，比文字描述格式和边界规则更不容易被误读；示例是最强的格式信号。
  3. 再答代价：每个示例都占上下文与费用；示例过多会让模型过拟合示例的表面特征（长度、措辞、顺序），还会把示例里无意带进去的偏见放大，比如四个示例里三个是 bug，它就更倾向判 bug。
  4. 结论：数量由「要覆盖几种类型」决定而不是越多越好，通常两到五个，每个覆盖一种不同的情况，并且至少一个是边界样本。
  5. 可预期的追问：示例的顺序有影响吗？有，多数模型对最后一个示例更敏感，所以把最像目标输入的放最后；另一个追问是示例和说明冲突时模型听谁的，答案是多半听示例，所以示例必须与格式栏逐字一致。

  Key points

  • Examples show the continuation directly, which beats prose for conveying format and edge rules
  • Too many examples cost context and money, overfit surface features, and amplify class bias
  • Pick the count by how many distinct cases need coverage, typically two to five with one boundary case
  • Order matters — put the closest match last; when examples and instructions conflict the model follows the examples

  答题要点

  • 示例直接展示下文该长什么样，比文字描述格式和边界规则更不容易被误读
  • 示例过多的代价：占上下文与费用、过拟合表面特征、放大示例里的类别偏见
  • 数量按「要覆盖几种不同情况」定，通常两到五个，至少一个边界样本
  • 顺序有影响，最像目标输入的放最后；示例与说明冲突时模型多半听示例

D3 Structured Output: JSON Schema, Templates and Variables, Multilingual Output

• Why should structured output be enforced with a schema instead of a 'please respond in JSON' instruction, and do you still need validation once the schema passes?结构化输出为什么要用 schema 约束，而不是在提示词里写「请输出 JSON」？schema 通过之后还需要校验吗？

  Common in ChinaCommon overseasBasic#structured-output#json-schema

  How to reason about it · think before answering

  1. This screens for whether the candidate has ever wired model output into code. People who only read output with their eyes think 'respond in JSON' is enough.
  2. List what that instruction cannot prevent: prose wrapped around the JSON, inconsistent key spelling, numbers as strings, extra keys, missing keys when an array is empty. Each maps to a schema keyword: required, enum, type, additionalProperties.
  3. Then the mechanism: the schema constrains generation itself, the model can only produce that shape, so the gain is qualitative rather than incremental.
  4. The second half is the differentiator: schemas constrain shape, not content — integer is not 4xx, string is not non-empty. Business rules still need code-level validation that returns an error list for retries.
  5. Follow-ups: strict-mode limits — every property in required, additionalProperties false, a supported subset of JSON schema, a first-use compile cost; and how to express optional fields — allow null in the type rather than dropping the key from required.

  分析过程 · 先想清楚再作答

  1. 这题在筛「有没有真的把模型输出接进过程序」。只在聊天窗口里用过模型的人会觉得「请输出 JSON」够了，因为他们是用眼睛读的。
  2. 拆法：列出「请输出 JSON」挡不住的几种踩空——外面包一段解释、字段名拼法不一致、数字变字符串、多出字段、空数组时省掉键。每一种都对应 schema 里的一个关键字：required、enum、type、additionalProperties。
  3. 再答原理：schema 在生成时约束形状，模型不是「生成完再检查」而是「只能生成这个形状」，所以稳定性是质变而不是量变。
  4. 后半句是区分度：schema 只能约束形状，不能约束内容——整数不等于 4xx，字符串不等于非空。业务规则必须在代码里再查一遍，校验函数返回错误列表供重试使用。
  5. 可预期的追问：严格模式有什么限制？所有字段都要进 required、要写 additionalProperties false、只支持 schema 子集、首次编译有开销；以及「可选字段怎么表达」——类型允许 null 而不是从 required 里去掉。

  Key points

  • 'Respond in JSON' only guarantees JSON, not which JSON: wrapper prose, key spelling, stringified numbers, extra or missing keys all slip through
  • A schema constrains generation itself; required, enum, type and additionalProperties each block one failure class
  • Validation is still needed after the schema passes because correct shape does not mean correct content
  • Strict mode needs every property in required and additionalProperties false; express optional fields by allowing null

  答题要点

  • 「请输出 JSON」只约束「是 JSON」，挡不住包解释文字、字段名不一致、数字变字符串、多字段、省键这几种踩空
  • schema 在生成时约束形状：required、enum、type、additionalProperties 各挡一种错误
  • schema 通过之后仍要校验业务规则，因为形状正确不等于内容正确
  • 严格模式要求所有字段进 required 且 additionalProperties 为 false；可选字段用允许 null 表达

D4 Iteration and Evaluation: Small Test Sets, A/B Testing, Version Control, Common Anti-Patterns

• How do you build a test set for a prompt? How would you choose ten samples, and where do the expected answers come from?怎么给一个提示词建测试集？十条样本该怎么挑，标准答案从哪来？

  Common in ChinaCommon overseasBasic#evaluation#test-set

  How to reason about it · think before answering

  1. This screens for whether the candidate has actually built one. 'Collect some inputs and run them' means no; people who have start with distribution, because prompt errors cluster at the edges.
  2. Three classes with three or four each: normal inputs guard the baseline; edge inputs (missing defaults, optional fields, informal phrasing) test whether default rules are explicit; adversarial inputs (distractors, mid-sentence corrections, unrelated asks) test focus. Add one or two unknowable items to check honesty.
  3. Expected answers are labeled by hand, no shortcut; one mislabeled case skews the whole evaluation and sends you chasing a phantom prompt bug. Re-read each input after labeling to confirm the answer is unique.
  4. Conclusion: ten is enough to start, value lies in distribution not count, and the best source is every real 'it failed again' input from the past week.
  5. Follow-ups: how does the set grow? Add the triggering input before every prompt change. And leakage — test cases must not double as few-shot examples, or you are measuring memorization rather than generalization.

  分析过程 · 先想清楚再作答

  1. 这题在筛「有没有真的建过测试集」。答「多找一些输入跑一跑」的人没建过；建过的人第一句会说分布——因为提示词的错误全集中在边界上。
  2. 拆法：三类各占三四条。正常输入守底线，新版弄坏它们就是严重回退；边界输入（没写默认值、可选字段、不规范写法）测默认规则说清没说清；刁难输入（干扰信息、中途改口、夹带无关要求）测能不能抓住重点。可以再放一两条模型不可能知道的样本，看它是否老实说不知道。
  3. 标准答案只能人工标，这一步没有捷径；标错一条整份评估就偏，而且你会误以为是提示词的问题去反复改。每条写完再读一遍输入确认答案唯一。
  4. 结论：十条够起步，价值在分布不在数量；最好的来源是过去每一次「它又错了」的真实输入，一周就能攒出比想象出来的更真实的测试集。
  5. 可预期的追问：测试集怎么增长？每次想改提示词先把触发的那条输入加进去再改；以及「测试集会不会泄漏进提示词」——用例不能直接当 few-shot 示例，否则是在测记忆而不是泛化。

  Key points

  • Distribution over count: three or four each of normal, edge and adversarial, plus a couple of unknowable items
  • Normal cases guard the baseline, edge cases test defaults, adversarial cases test focus
  • Expected answers are hand-labeled and re-checked; one wrong label skews everything
  • Best source is real failures; add the triggering input before each prompt change

  答题要点

  • 价值在分布不在数量：正常、边界、刁难三类各三四条，再放一两条模型不可能知道的
  • 正常输入守底线，边界测默认规则，刁难测抓重点
  • 标准答案人工标注、逐条复核，标错一条整份评估就偏
  • 最好的来源是真实出错的输入；每次想改提示词先把那条加进测试集