When should you use retrieval-augmented generation, when should you fine-tune, and when is stuffing the documents into the context window good enough?

One rule: retrieval for missing knowledge, fine-tuning for a missing way of speaking. Retrieval updates instantly by editing files, supports citation, and costs scale with the retrieved passages rather than the corpus. Fine-tuning is good at locking in style and output schema, poor at loading facts, and offers no traceability. Long-context stuffing wins when the corpus is small, stable and queried infrequently; it loses on cost and on locating facts once the corpus grows. If the answer does not depend on any document, use none of them.

In BM25, what problems do term-frequency saturation and document length normalisation each solve? What happens if you set both k1 and b to zero?

k1 controls saturation and prevents keyword stuffing: the score approaches a ceiling rather than growing linearly with frequency. b controls length normalisation and stops long documents from winning by sheer word count. Setting k1 to zero degenerates the scorer into boolean matching; one occurrence scores the same as a hundred. Setting b to zero removes document length from the equation entirely; both at zero leaves only a sum of IDF terms. IDF is the third component: it up-weights rare terms and removes the need for a stopword list.

A retrieval-augmented generation system gave a wrong answer. How do you determine whether retrieval or generation is at fault?

Always start by printing the retrieved passages and checking whether the correct answer is present at all. Split the pipeline into chunking, indexing, retrieval, context assembly and generation; diagnose right to left, fix left to right. Use symptoms to pin the stage: half-correct answers point at chunking, irrelevant hits at parsing, ignored material at the prompt, mismatched citations at generation. If retrieval missed the document, prompt changes cannot help; the material simply is not there. Log retrieved hits, the passages that entered the context, and the final answer together, or production failures are unreproducible.

Context windows are now in the millions of tokens. Does that make the retrieval step obsolete?

Separate whether it fits from whether it is worth paying for on every request. Small, stable, low-traffic corpora can legitimately be stuffed whole; building retrieval for them is over-engineering. Three reasons retrieval survives: per-request cost, corpora too large for any window, and attribution plus access control that must happen before the model sees the material. More supplied context reduces the reliability of locating a single fact, so stuffing everything is not automatically better. The trend is coarser retrieval — bigger chunks, more of them, less reranking pressure — not the removal of retrieval.

When are cosine similarity and inner product equivalent? What goes wrong if you rank by inner product on vectors that are not normalised?

Cosine equals inner product divided by both magnitudes; with unit magnitudes the divisor is 1, so they coincide. Without normalisation the inner product carries magnitude, and longer documents usually have larger magnitudes, biasing the ranking. The failure is silent, so normalise once at the embedding boundary and verify with offline evaluation. On normalised vectors L2 and cosine are monotonically related, so all operators rank the same. In pgvector the operators are ` ` for L2, ` ` for negative inner product and ` ` for cosine distance.

What do you lose when you cut embedding dimensions from 1536 to 512, and when is that loss acceptable?

Three costs: storage and index memory, query latency, and retrieval quality; the first two scale with dimensions, the third has diminishing returns. Matryoshka representations make truncation viable, but it is lossy and the loss must be measured on your own data. Decide by deriving a ceiling from the memory budget, then stepping down and measuring. Truncation pays off for large corpora, low-value items, latency-sensitive paths, and pipelines with a reranker. All vectors in one index share a dimension, so changing it forces a full rebuild.

Why do some embedding models require different prefixes for queries and documents? What happens if you skip them, and how would you catch it before shipping?

These models are trained on question-passage pairs; the prefix marks which role a text plays, and omitting it puts you off-distribution. Skipping prefixes never errors, it only degrades quality, so the failure is silent. The reliable detection is an A/B run over a small labelled question set, comparing hit rate. A subtler bug is mismatched or identical prefixes on both sides, which looks healthier but misaligns queries and documents. Keep prefixes inside the embedding call convention, and re-read the model card whenever you switch models.

Can vector search fully replace keyword search? Give a query where vectors are bound to fail, and say how you would fix it.

No: codes, version numbers, names and IDs matter as exact literals, which embeddings blur into neighbourhoods. Concrete example: asking whether rate limiting returns 429, where BM25 hits the document containing 429 and vectors surface a topically similar one that never mentions it. Negation is a second failure class, since affirmative and negative statements sit almost on top of each other. The remedy is hybrid retrieval: run both paths and merge with reciprocal rank fusion. Fusion pays off because the two paths fail differently; query rewriting helps vocabulary mismatch but not exact identifiers.

The text extracted from a PDF comes out in the wrong order. How do you diagnose and fix it?

State the premise: a PDF stores only drawing instructions, so paragraphs and reading order are inferred, not read. Debug by dumping fragments with page, coordinates and font size; plain text hides the cause. Three common causes: undetected multi-column layout, content stream written in drawing order, and headers or footers left in. Fix columns by finding the widest gap between left edges and sorting by column, then y descending, then x ascending. Add a ground-truth-free regression metric such as an out-of-order score so the fix stays fixed.

Which metadata should a document parsing stage preserve, and which downstream feature breaks if you drop each one?

The rule is recoverability: if it cannot be recovered from the original later, capture it at parse time. Chunk ids back verifiable citations, heading paths back localisation and structure-aware chunking, page numbers make citations land precisely. Access-control labels must be attached during parsing, otherwise enabling ACL means re-parsing everything, and teams end up filtering at generation time where the leak has already occurred. Updated-at lets you present conflicting sources side by side; a content hash enables incremental sync instead of full rebuilds. When unsure, keep the field: storage is far cheaper than a full re-parse.

OCR output from scanned documents carries a non-trivial error rate. How does that noise propagate into retrieval and generation, and how do you mitigate it?

Retrieval: visually similar characters break literal matching, and bigram tokenisation lets one bad character corrupt two tokens, so recall drops silently. Generation: the model reads through general noise but confidently repeats corrupted entities, and citation verification against a corrupted source proves nothing. At ingest: check for a text layer before running OCR at all, and keep the source image for human verification. At retrieval: hybrid search helps because dense retrieval tolerates a single wrong character better than literal matching. At generation: flag low-confidence sources in the answer, and restrict auto-correction to low-confidence spans while keeping the raw text.

Why is parsing quality the ceiling on retrieval quality? Walk through one concrete chain of propagation.

Parsing is stage zero, before the five-stage pipeline; its errors are amplified downstream and invisible to every later stage. Concrete chain: a shifted table, chunking on wrong boundaries, wrong term pairs in the index, that chunk ranked first, and a wrong answer delivered with a citation. The dangerous part is that nothing errors out and the answer carries a source, so it looks more credible than usual. Later techniques only improve selection from candidates; if the material is wrong, better selection still returns something wrong. Diagnose right to left: print the retrieved passages first, and if the source text is already broken, stop looking at the generation side.

How do you decide on chunk size? Name two metrics you would look at, and one counterexample.

Choose the strategy from the document type first, then tune length: split on headings whenever the structure survives parsing. Watch two metrics: retrieval hit rate on one side, mid-sentence break rate (then faithfulness and citation resolvability) on the other. Compare under an equal token budget, never a fixed top-k, or larger chunks win by buying more text. Counterexample: hit rate rises after enlarging chunks because whole documents now fit in one chunk and retrieval has effectively stopped working. Start near 300 to 500 characters with 10 to 20 percent overlap, then iterate against a fixed question set instead of guessing.

What does parent-child chunking buy you, and when does it slow the system down instead?

The core idea is decoupling the retrieval unit from the context unit: small chunks get found, large chunks get understood. The payoff is precision and completeness at the same time instead of trading one for the other. Cost one: a single hit can drag in a whole parent, so an equal context budget holds fewer distinct results and diversity suffers. Cost two: two index levels to maintain and recompute, which makes incremental sync on document updates considerably harder. It stops paying off when sections are already short, because parent and child are nearly identical and you bought nothing for the extra cost.

What overlap ratio would you use, and what concretely goes wrong when the overlap is too large?

Ten to twenty percent of chunk length is the working range, but that number assumes fixed-length splitting. With structural or recursive splitting the boundaries are already semantic, so overlap can be small or zero. Cost one: index tokens and storage grow noticeably; at 400-character chunks, an 80-character overlap adds roughly fifteen percent. Cost two: neighbouring chunks become near-duplicates, so the top results are several versions of one passage and the evidence diversity is illusory. Cost three: a sentence spanning two chunks complicates citation attribution and forces result-level deduplication.

Semantic chunking costs considerably more than recursive splitting. How would you prove to your team that the money is well spent?

Start with the precondition: the gain comes from documents without usable structure, so on well-formed documents it usually is not worth it. Translate 'worth it' into three numbers — hit rate, latency, and cost. Reporting only the first is not an argument. Run a controlled comparison: same corpus, same golden set, same context budget, same retriever, with the splitting strategy as the only variable. Fix the decision threshold before running the numbers so you cannot bend it to fit the result afterwards. Try free structural splitting first, and if semantic chunking is genuinely needed, apply it to the high-value subset rather than the entire corpus.

How do you choose between an HNSW index and an IVFFlat index? Give one scenario that forces each choice, and name the parameter you would tune first in each.

HNSW is a layered neighbour graph; IVFFlat clusters first and scans a subset of lists. HNSW favours query quality, IVFFlat favours build cost and memory. Tune the query-side knob first: ef_search for HNSW, probes for IVFFlat. Neither needs a rebuild. Heavy write traffic with tight memory and build windows points to IVFFlat; a static corpus with a hard latency SLA points to HNSW. IVFFlat clusters drift with the data and need scheduled rebuilds; HNSW does not, but its index is often larger than the table. Know the defaults: probes 1, ef_search 40. Leaving probes at 1 wastes the index.

Why does a vector search with a WHERE clause return fewer results than expected, and what are the fixes and their costs?

With approximate indexes the filter runs after the index scan, so a selective condition wipes out most candidates and the query returns too few rows. There are two failure modes: too few rows, and enough rows in the wrong order. Always check the returned count first. Iterative scanning fixes only the first. Strict ordering preserves distance order, relaxed ordering gives better recall, and both raise latency noticeably. Pre-filtering is the alternative: index the filter column for exact search, use a partial index for a few distinct values, partition by value for many. Costs are losing the approximate speedup, index sprawl, and operational complexity. The second failure mode is only fixed by raising probes or ef_search; iterative scanning does nothing for it.

If you switch your vectors from full precision to half precision or binary quantisation, how do you verify that recall has not dropped materially?

Ground truth must come from an exhaustive scan with indexes disabled; using index output as truth pins recall near 100%. Run one fixed query set before and after, report recall at k together with index size, build time and latency percentiles. Quantisation loss depends on your own vector distribution, so measure it on your data instead of quoting benchmarks. Half precision is usually near lossless and raises the indexable dimension limit from 2000 to 4000, making it a safe default. Binary quantisation loses real recall; use it as a cheap first pass and re-rank with the original vectors over a wider window.

When should you move your vectors out of PostgreSQL into a dedicated vector database? Give measurable triggers, and also make the case for staying.

Default to staying in PostgreSQL: transactions, backups, recovery, permissions, joins and familiar tooling are free, and a second store adds sync and on-call cost. Trigger one is data volume, measured by whether the index still fits in memory rather than by row count. Trigger two is write frequency: minute-level streaming updates distort clusters and inflate graphs. Trigger three is filter complexity: dedicated stores push filtering into the index structure, a mechanical advantage under complex predicates. Trigger four cuts the other way: without people to run a second database, do not move even if the first three hold. Often hybrid retrieval plus re-ranking is the real fix.

How do you make sure a model's citations are real rather than fabricated? Describe a scheme that does not rely on the model behaving well.

Citations must be closed-set symbols such as block numbers, not free-text titles: verifiability comes from the closed set, not from wording. Two gates: the number must exist, and the sentence must substantively overlap the cited block's original text, which is what catches real-number-wrong-content fabrication. Strip terms that occur in most blocks before scoring overlap, or any citation will pass. On failure, regenerate once with the concrete reason fed back; two bad drafts means refuse instead. Always verify against the original text the user can open, never against a compressed or rewritten copy.

Does the ordering of retrieved passages in the context affect answer quality? If so, how would you order them?

Yes: material at the head and tail is used more reliably, the middle is most often skipped. Put the strongest at both ends: rank one first, rank two last, rank three second, folding inward. Assembly also needs a deterministic tiebreaker for stable numbering, dedupe on normalised text, and a token budget that skips oversized blocks instead of stopping. The strength of the effect varies by model and context length, so measure it on your own data instead of quoting it as a law.

How do you set the refusal threshold for a knowledge-base assistant, and what does it cost you when the threshold is too high or too low?

Refusal is three rules, not one: low score and source conflict decided before generation, out-of-coverage decided after generation from the verification result. On conflict, presenting both versions versus picking the newer one is a product decision; picking only holds up when an authoritative signal backs it. The three responses must be worded differently because each implies a different next action for the user. Too high blocks answerable questions; the user is told nothing exists while it does, which is the most damaging and least visible failure. Too low lets weak passages in, producing errors that are harder to spot because the answer still carries citations. Set it by comparing score distributions over answerable and unanswerable question sets, then choose based on which error is worse; re-tune whenever the corpus or retriever changes.

In a streaming setup, how do you make sure nothing you have already sent needs to be retracted because its citation failed verification?

The conflict: emitted text cannot be recalled, while a citation can only be checked once its sentence is complete. The fix is sentence-level buffering: verify each completed sentence, emit only if it passes, drop the whole sentence if it does not. The cost is time-to-first-sentence instead of time-to-first-token, which is affordable and worth paying. Streaming cannot use JSON, so use inline markers in plain text while sharing one verifier with the non-streaming path; strip markers from the prose and send numbers as structured data. Emit pre-generation refusals before the stream opens; the post-generation one manifests as an empty stream and is closed with a refusal event.

How would you draw the module boundaries of a RAG system, and which layer most needs to be swappable? Why?

Lead with the criterion: cut where a layer is most likely to be replaced wholesale. The embedding layer is the one to abstract: swapping models invalidates every stored vector and forces a full recompute. Storage is the single boundary shared by ingestion and query, so define its interface before either implementation. Chunking and retrieval routes belong in configuration because they change most often during tuning. Abstraction costs indirection, so only abstract changes that will actually happen.

What should the ingestion path and the query path share, and what concretely goes wrong when you over-share?

Share the interface, not the flow: storage is the only boundary, plus the embedding signature. The two paths have different error handling and latency budgets; batch can rerun, online must fail fast. Over-sharing shows up as isIngest branches and changes that must be verified twice. What must match is the model choice, not the code: record the model name alongside every stored vector. Chunking belongs to ingestion only; the query path reads larger units back through storage.

What three checks would you run before shipping a retrieval QA service, and why those three?

State the ranking rule first: prioritise failures users cannot see but that cost the most. Check one, verifiable citations: every id resolves to a real chunk that overlaps the sentence citing it. Check two, refusal actually fires on a question the corpus cannot answer. Check three, ingestion and retrieval agree: new documents are immediately retrievable on both routes. Latency and cost matter but rank lower because those failures announce themselves.

What is the biggest risk in the RAG service you just assembled, and how would you prove that judgment?

The biggest risk is having no evaluation: every parameter was guessed, so no change can be judged. Prove it by building a golden set with known answer documents, including unanswerable and multi-hop questions, then baseline the current configuration. Validate the set itself by perturbing parameters: metrics that never move mean the questions are wrong. Report three numbers per optimisation: metric gain, added latency, added cost. Start small but well covered, then grow toward the real question distribution.

You need to build an evaluation set from scratch for a RAG system over a company knowledge base. How would you do it, and how many questions are enough?

Author corpus-first so the answer document is known at authoring time. Label every item with a type: single-document, multi-hop, unanswerable. Multi-hop requires all answer documents; unanswerable items score abstention, not recall. 20 items for a smoke gate, 100 to 200 to trust small deltas, and keep growing it from production failures. Hold out a slice that never informs tuning to avoid overfitting the set.

Recall, mean reciprocal rank, and normalized discounted cumulative gain - which failure mode does each one catch first, and what do you miss by watching only one?

Recall answers 'did it make it into the context', sensitive to total misses, blind to rank shifts. MRR answers 'how high is the first hit', sensitive to ranking degradation, blind to how many hits there are. nDCG answers 'how good is the top k overall', the direct target for reranking, blind to existence. Only the combination localizes the failure to retrieval, ranking, or noise. State the hit criterion: context is packed against a token budget, not a fixed top-k.

What systematic biases does an LLM judge have when scoring RAG faithfulness, and how do you detect them and prove your judge is trustworthy?

Three biases: position, verbosity, and self-preference, each with a matching mitigation. Score pointwise rather than pairwise; decompose into claims and score a ratio to kill the length premium; never let the generator judge itself. Pin rubric anchors in the prompt and force structured output that quotes unsupported sentences. Establish trust through stratified human spot-checks and an agreement rate; below 0.8 the judge cannot gate merges. Add probes with known verdicts so a broken judge is caught on every run.

Why must a RAG evaluation set include questions the corpus cannot answer, and what does leaving them out hide?

An all-answerable eval set makes 'answers confidently when it should not' completely invisible. Unanswerable items are scored on abstention, not recall, and you check both the retrieval gate and the generation refusal. Author them with strong distractor terms, or retrieval returns nothing and you are testing the tokenizer. Keep them at roughly 15% or more of the set, alongside multi-hop items, as the coverage floor. A low abstention rate splits into two causes: a no-op retrieval score gate, or a missing refusal instruction in the prompt.

Why do hybrid retrieval systems usually use reciprocal rank fusion instead of normalizing both scores and adding them with weights? When does the weighted approach break down?

BM25 is unbounded, cosine is bounded; the two scales are not comparable, so adding them is meaningless. Per-route max normalization has a denominator that floats with the query, so the least relevant hit of an unanswerable query still normalizes to 1.0. Weights must be tuned against an eval set, the search is high-dimensional once you add routes, and swapping models invalidates it; RRF has a single constant k. RRF consumes only the ordered id list from each route, because rank is the one thing the routes share. Larger k rewards `ranked well by both routes`; ties must fall back to document id so results are reproducible.

Why is a cross-encoder more accurate than a bi-encoder? And if it is more accurate, why not just use it to search the whole corpus directly?

A bi-encoder encodes both sides separately and joins them with one dot product; a cross-encoder concatenates them so attention can align across the pair. The accuracy gap comes from compression: a bi-encoder squeezes a whole document into one vector and loses bindings; a cross-encoder does not compress. Bi-encoder document vectors can be computed offline and indexed; a cross-encoder has nothing to precompute. Reranking the full corpus means running every chunk through a model on every question, so cost scales linearly with corpus size. The standard split is recall plus rerank, with reranking applied only to the top few dozen after fusion.

You replaced pure vector retrieval with hybrid search plus reranking, and after shipping it your eval metrics went down. How do you investigate?

Run all four configurations separately for attribution, on one eval set with one context budget, before touching any parameter. Separate a recall drop from a ranking drop: the first is a candidate-pool or budget issue, the second is a fusion or rerank issue. Check recall depth: taking too many per route lets noise vote, and narrowing it can bring recall back. Confirm the eval protocol did not change; touching budget, hit rule, or candidate depth makes old and new numbers incomparable. Freeze the four-way comparison into one command plus a baseline report, and block merges on regression.

Adding a reranker costs you 200 ms of extra latency per question plus a per-search fee. How do you decide whether that spend is worth it?

Report all three ledgers together: metric gain, latency growth, cost growth; a proposal missing one is incomplete. Confirm whether the bottleneck is ordering or recall first; reranking only reorders and cannot rescue an answer that was never retrieved. Judge the latency by where it lands: it is small relative to a streaming generation, but fatal in an as-you-type search box. Rerankers bill per search rather than per token, so cost scales with question volume, not candidate count. If it is unaffordable: route only hard queries to the reranker, cache results, or self-host an open-weights cross-encoder.

Why does HyDE (hypothetical document embeddings) work, and when does it steer retrieval in the wrong direction?

It works by register alignment: a question and a policy paragraph sit in different neighbourhoods, and the fake passage moves the query into the document's. The draft's factual accuracy is irrelevant — it only supplies a direction and is never shown to the user. It misfires when the model invents over-specific details, or when the corpus has no answer and the fabrication finds plausible neighbours anyway. Two guardrails: gate on raw per-route scores, not fused ones; fuse the hypothetical document with the original query instead of replacing it. It is the most expensive query-side technique (long output plus doubled retrievals), so keep it off by default and A/B it.

How do you handle coreference in multi-turn RAG, and what is the classic failure when you skip it?

Add a short rewrite call before retrieval: last few turns plus current question in, one retrieval line out, temperature 0, answering explicitly forbidden. Do not splice the whole history into the query — it grows unbounded, dilutes IDF, and leaks the previous answer into the search. Classic failure: an unresolved pronoun means no candidate clears the gate, so the system refuses a question the corpus can answer. That false "not found" hurts more than a wrong answer, since the user just asked about the same thing. Order matters: rewrite first, then route; multi-query and step-back both build on the rewritten query.

Query rewriting adds a model call per question and doubles end-to-end latency. How do you decide whether it is worth paying?

Locate the gain first: rewriting is near-zero on single-turn traffic and pays off on follow-ups, so start from the share of multi-turn sessions. All three ledgers are mandatory: metric delta on a golden set, added latency, added calls and token cost. Try the cheap variants before deciding: conditional triggering, per-session caching, and a small model for the rewrite. Decide on gain times affected traffic share over added latency and cost, then rank it against your other optimisations. If latency is a hard constraint, fire the rewrite in parallel with the first retrieval and fuse both rankings, turning a sum into a max.

What happens when intent routing misclassifies, and how would you design the fallback?

The three routes have asymmetric error costs: sending a retrieval-worthy question to a direct answer is the worst, while routing chit-chat to single-hop only wastes one retrieval. Bias the fallback toward spending more: default to single-hop whenever the classifier is unsure, since that error is the mildest and is recoverable. Add runtime fallbacks: re-retrieve if a direct answer contains figures that need a source; escalate or abstain if no single-hop candidate clears the gate. Log every routing decision — raw question, label, whether a fallback fired — for both monitoring and the next training set. When chit-chat and multi-hop are both rare, the classification call costs more than it saves; route everything to single-hop instead.

Parent-child indexing and contextual retrieval both patch the same problem — chunks losing their context. What actually distinguishes them?

Parent-child acts on the generation side: retrieve small, swap in the parent for context. It fixes 'found but unreadable'. Contextual retrieval acts on the retrieval side: the header makes the chunk findable. It fixes 'readable but never found'. They are orthogonal and compose; keep the header in the index only, never in the context window. Parent-child costs more index entries and a larger context unit; contextual retrieval costs one call per chunk plus a permanently larger index. Pick based on the observed failure: incomplete evidence points to the former, zero retrieval to the latter.

Contextual retrieval needs one model call per chunk. How do you estimate that one-off cost, and what levers bring it down?

Split into one-off (per-chunk input/output plus re-embedding) and per-query (header read by both reranker and generator). The one-off is dominated by re-reading each document n times; caching turns that into one write plus n-1 reads. Caching is prefix-matched: the full document must come first, the chunk after, or you get zero hits. Measured on 30 docs / 134 chunks, caching cut the one-off cost by about 29%, and finer chunks save more. The lasting cost is per query: keep headers out of the context window, keep them short, and generate them selectively.

What kind of question actually requires graph retrieval? Give one concrete case where it is justified and one where it is not.

The test is not 'is it multi-hop' but 'does the second document overlap the query at all' — only zero overlap earns a graph. Justified: the approver question, where an intermediate entity is the only bridge and the second doc never enters the candidate pool. Not justified: a multi-hop question whose documents both overlap the query — hybrid retrieval collects them in one pass. Real graph cost is entity disambiguation, relation dedup, incremental subgraph recomputation and a whole extra store — not a single extraction call. Try two-pass agentic retrieval first; build the graph only when that fails.

You have built three different indexes over the same corpus. How do you decide which one a query goes to?

First challenge the premise: all five index structures landed at the same 93.8% recall in our measurement, so an index without a matching weakness is pure cost. At request time the usable signal is question shape — detail, summary, or entity-chaining — mapping to chunk, tree-summary and graph indexes. Reuse the previous day's intent router for classification and record the routing decision as request metadata. Fall back to the default index on misclassification instead of fanning out and fusing, which multiplies latency and cost. Replay the golden set periodically to check whether the classifier picks the best-scoring index.

You are exposing retrieval to a model as a tool. How do you write the tool description, and what concrete failure modes appear when you write it badly?

The description is a prompt for the model, not a code comment: scope, when to call, when not to call, query shape. Missing scope turns it into a web search; missing 'must call' produces confident answers from memory. Missing 'do not call' makes small talk trigger retrieval, paying cost and latency for nothing. Stating 'keyword phrase, no question words' fixes query pollution at the source. Optional filters need 'only set when certain' - a wrong filter silently hides the right answer. Regression-test with a negative suite and assert the tool was not invoked.

Self-reflective retrieval rewrites the query and retries. How do you guarantee it terminates instead of spinning on the same query forever?

Three brakes, none optional: max rounds, cumulative token budget, duplicate-query detection. The cumulative budget catches rounds that each pass but blow up together - the round cap cannot see that. Check for duplicates before retrieving, and normalize the query to a term set before comparing. Record stop reasons as distinct categories rather than one 'finished' bucket. Every brake needs a case that actually trips it; an untested brake is no brake. Have the assessor emit covered and missing elements so 'not enough' is auditable.

In multi-hop retrieval a wrong first hop poisons every hop after it. How would you design for that?

Classify first: never retrieved needs a new query term (a hop); retrieved-then-filtered needs a relaxed gate. The fixes are opposite. Derive the next query from a bridge phrase - a concrete noun absent from the question that also appears in another document. Keep the previous hop's material so a failed hop does not discard existing evidence. Trace every hop separately so you can pinpoint which one drifted. Allocate context budget round-robin across hops, or a successful hop still fails to change the metric. Surrender explicitly when no lead remains and hand it to the generation-side refusal.

When would you refuse to make a RAG system agentic, and what data would you use to convince your team?

Gains concentrate in multi-hop and retry cases; single-document questions gain almost nothing. Settle it with the evaluation set: multi-hop delta against the multiplier on total calls. One measured set: multi-hop recall 75% to 100%, overall 93.8% to 100%, retrieval calls 1 to 1.75 plus the same number of assessment calls. Refuse when latency-sensitive, when question patterns are fixed, or when cost is tight - variance hurts more than the mean. Route instead: a cheap check up front, and only multi-hop-looking questions enter the loop. Looping is a capability, not a default.

After a document changes, how do you recompute only the affected chunks? And how do you guarantee a deleted document really disappears from the index?

Three-way reconciliation covering adds, modifies and deletes; a modify replaces the whole document, old chunks first. Normalize line endings and trim before hashing, or cross-platform re-uploads look like edits and you are doing a full rebuild every night. Deletion is an absence, not an event: walk the index for ids the source no longer has instead of waiting on a change feed. Cascade deletes from documents to chunks to embeddings so one statement suffices; assert chunk count equals embedding count to catch orphans. On an incomplete pull, apply adds and modifies only, and soft-delete with a retention window so mistakes are reversible.

Why can't access control be applied at the generation step? What exactly leaks if you put it there?

Filtering at generation time means unauthorized documents were already retrieved, ranked, held in memory and written to logs and traces; the exposure is far wider than 'did the text reach the answer'. The second consequence is dilution: filtered-out slots are not backfilled, so users see 'nothing found' while the log shows a normal retrieval. The fix is to put the permission predicate in the same statement as ordering and LIMIT so the database prunes before ranking. Choose between row-level filtering and index isolation by the count and stability of the boundaries; a shared index lets a large tenant crowd out a small one's candidate slots. Caching is the same bug's second crime scene: the answer cache key must carry the permission scope or answers leak across users without a trace.

What can be cached in a RAG system, and what are the invalidation conditions for each?

Three layers — answer, retrieval, embedding — with lifetimes orders of magnitude apart; treating them as one thing is the mistake. The rule is that 'when must it expire' equals 'is it in the key'; anything left out of the key can never invalidate the entry. The answer key carries question, permission scope, index version, model and prompt version; the retrieval key drops the model and adds topK and the embedding backend. The embedding layer is content-addressed and keyed only on text plus backend; adding an index version turns every sync back into a full rebuild. Version-based invalidation beats targeted deletion because you cannot enumerate which questions a given change affected.

Interview Bank

328 questions total; 57 shown with current filters.

CourseAll From Frontend Engineer to Agent Engineer in 30 Days Prompt Engineering From Scratch in 5 Days Mastering Claude: From Conversation to Claude Code in 5 Days Mastering Codex and the OpenAI Agents SDK in 5 Days MCP in 7 Days: Wire Tools Into Any Agent Agent Skills in 7 Days: Turn Experience Into Reusable Capability Context Engineering in 5 Days RAG in 14 Days: From Retrieval to Trustworthy Answers Build an AI Short-Drama Production Pipeline With Agents in 14 Days

WeekAll W1 W2

MarketAll Common in China Common overseas

DifficultyAll Basic Intermediate Deep dive

Tag

All #evaluation15 #chunking6 #cost6 #embeddings5 #agentic-rag4 #architecture4 #hybrid-search4 #ingestion4 #abstention3 #data-quality3 #access-control2 #citation-verification2

92 more tags

#contextual-retrieval2 #cost-tradeoff2 #debugging2 #failure-modes2 #golden-set2 #long-context2 #multi-hop2 #observability2 #query-rewriting2 #ranking2 #recall2 #refusal2 #retrospective2 #system-design2 #api-design1 #bi-encoder1 #bm251 #caching1 #citations1 #content-hash1 #context-assembly1 #coreference1 #cost-optimization1 #cross-encoder1 #dimensions1 #embedding-migration1 #engineering-judgement1 #error-propagation1 #evidence1 #failure-analysis1 #faithfulness1 #fallback1 #filter-pushdown1 #filtering1 #fine-tuning1 #graph-rag1 #grounding1 #hallucination1 #hnsw1 #hyde1 #incremental-sync1 #index-maintenance1 #index-routing1 #indexing1 #information-retrieval1 #intent-routing1 #invalidation1 #iterative-scan1 #ivfflat1 #latency1 #latency-budget1 #llm-as-judge1 #metadata1 #model-selection1 #modularity1 #multi-tenancy1 #multi-turn1 #normalisation1 #ocr1 #ordering1 #overlap1 #parent-child1 #pdf-parsing1 #prioritization1 #production-readiness1 #prompt-caching1 #prompt-engineering1 #prompting1 #quantization1 #query-transformation1 #rag1 #rag-basics1 #rank-fusion1 #reliability1 #rerank1 #retrieval1 #retrieval-failure1 #retrieval-metrics1 #retrieval-quality1 #risk-assessment1 #rollout1 #scaling1 #self-reflection1 #similarity1 #stakeholder-communication1 #streaming1 #thresholds1 #tool-design1 #trade-offs1 #vector-database1 #vector-index1 #zero-downtime1

RAG in 14 Days: From Retrieval to Trustworthy Answers

D1 Why Retrieve at All: Hallucination, Knowledge Cutoffs, and the Cost of Long Context; a Minimal Keyword-Only RAG

When should you use retrieval-augmented generation, when should you fine-tune, and when is stuffing the documents into the context window good enough?什么时候该用检索增强生成，什么时候该微调，什么时候直接把文档塞进上下文就够了？
Common in ChinaCommon overseasBasic#rag-basics#fine-tuning#long-context
How to reason about it · think before answering
This question shows up in almost every loop. The differentiator is not reciting three definitions, it is offering a decision rule the interviewer can reuse.
Lead with the rule: is the model missing knowledge, or missing a way of speaking? Missing knowledge means retrieval; missing style or output shape means fine-tuning. That single cut covers most cases.
Then line up the three options against three costs: cost of updating knowledge, cost per request, and whether the answer can be traced back to a source. Retrieval updates by editing a file, fine-tuning takes a retraining cycle, and long-context pays for the whole corpus on every call.
Give long-context its fair case: when the corpus is small, changes rarely, and request volume is low, stuffing it in is the cheapest engineering decision you can make. It stops being cheap once the corpus grows or the same material is queried thousands of times a day.
Close by naming when none of this applies: if the answer does not depend on any external document (rewriting, translating, reformatting), retrieval only adds noise, latency and cost.
Expected follow-up: can you do both? Yes, and it is common. Fine-tuning controls format and refusal behaviour, retrieval supplies the facts.
分析过程 · 先想清楚再作答
这题几乎每场都问，区分度不在能不能背出三条定义，而在你会不会给一条判据。只说「RAG 适合动态知识、微调适合特定风格」的人一抓一大把，面试官等的是下一句。
先给一条能当场套用的判据：模型缺的是「知道什么」还是「怎么说」。缺知识走检索，缺风格与输出格式走微调，这一刀切下去能分掉八成场景。
再拿三笔账把三条路排开：知识更新的代价（改文件立刻生效 / 重训以天计 / 改文件立刻生效）、单次成本（只付取回的几段 / 只付推理 / 每次都付全量材料）、能不能归因（能 / 不能 / 能但材料一多定位会飘）。
把上下文直塞的适用边界说清楚：材料总量小、更新不频繁、对单次成本不敏感的场景它最划算，因为工程量近乎为零。一旦材料涨到几百篇，或者同一批材料每天要被问上万次，成本曲线立刻反超。
最后主动补一句「什么时候都不该用检索」——任务的答案不依赖任何外部文档时（改写、翻译、格式转换），加检索只会引入噪声、延迟和成本。能主动划出不该用的边界，比会背适用场景更能证明你做过。
可预期的追问：能不能既微调又检索？答案是可以，而且常见——微调管输出格式与拒答口径，检索管事实，两者解决的不是同一个问题。
Key points
One rule: retrieval for missing knowledge, fine-tuning for a missing way of speaking.
Retrieval updates instantly by editing files, supports citation, and costs scale with the retrieved passages rather than the corpus.
Fine-tuning is good at locking in style and output schema, poor at loading facts, and offers no traceability.
Long-context stuffing wins when the corpus is small, stable and queried infrequently; it loses on cost and on locating facts once the corpus grows.
If the answer does not depend on any document, use none of them.
答题要点
一条判据：缺「知道什么」用检索，缺「怎么说」用微调。
检索改文件即时生效、可归因、成本只跟取回的几段有关，代价是要自己建一套会出错的检索系统。
微调擅长固化风格与输出格式，不擅长灌事实：数据一变就要重训，而且没法归因。
长上下文直塞在小型、低频、少变的语料上最划算，材料变多或调用量变大之后成本与定位稳定性都会恶化。
任务答案不依赖外部文档时三条路都不该用，直接调模型。
In BM25, what problems do term-frequency saturation and document length normalisation each solve? What happens if you set both k1 and b to zero?BM25 里的词频饱和与文档长度归一化分别在解决什么问题？把 k1 和 b 都设成 0 会发生什么？
Common in ChinaCommon overseasIntermediate#bm25#ranking#information-retrieval
How to reason about it · think before answering
This checks whether you have actually read the formula rather than merely called a library. The test is whether you can map k1 and b onto specific terms and name the failure each one prevents.
Start with the two holes in raw term frequency: keyword stuffing lets one document dominate by repeating a word, and long documents win by accident because they contain more words overall.
k1 closes the first hole. Term frequency appears in both numerator and denominator, so the ratio approaches a ceiling instead of growing linearly. Fifty mentions are more relevant than five, but not ten times more relevant. A smaller k1 saturates sooner.
b closes the second. The normalisation factor is one minus b plus b times document length over average length: at b equal to zero length is ignored entirely, at one it is fully penalised, and 0.75 is the conventional compromise.
Now the trap in the question: k1 equal to zero collapses the ratio to a constant, so one occurrence scores the same as a hundred and matching becomes boolean. b equal to zero removes length entirely. Set both to zero and BM25 degenerates into a plain sum of inverse document frequencies.
Expected follow-up: can you drop the IDF term? No. Without it, ubiquitous words drown everything else, and it is precisely IDF that lets BM25 work without a stopword list.
分析过程 · 先想清楚再作答
这题考的是你有没有真的读过公式，而不是有没有调过库。判据很明确：能不能把 k1 和 b 各自对应到公式里的哪一项，并说出去掉之后会被什么样的文档钻空子。
先说朴素词频的两个漏洞：一是重复刷词，一篇文章把关键词写五十遍就能霸榜；二是长文占便宜，文档越长越容易蒙中查询里的词。这两个漏洞正好对应两个修正。
k1 管第一个漏洞。分子分母里都有词频 f，所以词频涨上去之后整个分式趋近一个上界而不是线性增长——写五十遍确实比写五遍相关，但绝不该相关十倍。k1 越小饱和越快。
b 管第二个漏洞。归一化项是 1 减 b 加上 b 乘以本文长度除以平均长度，b 等于 0 时完全不看长度，b 等于 1 时完全按长度比例惩罚，0.75 是长期折中的默认值。
回到题干那个陷阱：k1 设成 0 会让分式退化成常数，词出现一次和一百次得分完全一样，等于只剩「有没有出现过」的布尔匹配；b 设成 0 则长度信息彻底消失。两个一起设成 0，BM25 就退化成对逆文档频率求和，跟词频再无关系。
可预期的追问：那逆文档频率去掉行不行？答案是不行，去掉之后「的」「我们」这类高频词会淹没一切——而且要顺带说明 BM25 因此天然不需要停用词表，这一句最能体现你读懂了公式。
Key points
k1 controls saturation and prevents keyword stuffing: the score approaches a ceiling rather than growing linearly with frequency.
b controls length normalisation and stops long documents from winning by sheer word count.
Setting k1 to zero degenerates the scorer into boolean matching; one occurrence scores the same as a hundred.
Setting b to zero removes document length from the equation entirely; both at zero leaves only a sum of IDF terms.
IDF is the third component: it up-weights rare terms and removes the need for a stopword list.
答题要点
词频饱和由 k1 控制，防的是重复刷词：词频涨大后得分趋近上界而非线性增长。
长度归一化由 b 控制，防的是长文档靠词多蒙中查询，用本文长度比平均长度把它压回去。
k1 设 0 会退化成布尔匹配，词出现一次和一百次同分；b 设 0 则完全不考虑文档长度。
两者都设 0 时 BM25 只剩逆文档频率求和，等于放弃了词频信息。
逆文档频率是第三块，让稀有词权重更高，也让 BM25 天然不需要停用词表。
A retrieval-augmented generation system gave a wrong answer. How do you determine whether retrieval or generation is at fault?一个检索增强生成系统答错了，你怎么定位是检索的锅还是生成的锅？
Common in ChinaCommon overseasIntermediate#debugging#failure-modes#evaluation
How to reason about it · think before answering
The question asks how you localise the fault, not what the possible causes are. Listing causes loses; the interviewer wants an ordered procedure that ends in concrete actions.
Give the cheapest first step: print the retrieved passages verbatim and read them. If the correct answer is not in there, retrieval is at fault. If it is in there and the model ignored it, generation is at fault. Thirty seconds, and it removes most of the guesswork.
Then lay out the five stages — chunking, indexing, retrieval, context assembly, generation — with the rule: diagnose right to left, fix left to right. You see the generated answer first, but an error on the left is amplified by everything to its right.
Add symptoms that pin down a stage: half-correct answers usually mean a rule was split across chunks; obviously irrelevant hits usually mean dirty parsing; the model ignoring the supplied material usually means the prompt never said it must; citation numbers that do not match their content point at generation.
Land it in engineering terms: to run this procedure repeatedly you must log the retrieved hits, the passages that entered the context, and the final answer together, otherwise production issues are unreproducible. At scale this becomes a fixed question set with metrics rather than case-by-case reading.
Expected follow-up: if retrieval missed the document, will prompt tuning help? No. Nothing in the prompt can conjure material that was never supplied.
分析过程 · 先想清楚再作答
题眼在「怎么定位」，不在「有哪些原因」。答成一串可能原因的罗列就输了，面试官想听的是一个有先后顺序、能落到具体动作的排查流程。
先给最省时间的第一步：把这次检索出来的几段原文原样打印出来，自己读一遍。正确答案不在里面就是检索的锅，在里面而模型没用上才是生成的锅。这一步三十秒，能省掉大半天的瞎猜。
然后把链路展开成五个环节——切块、建索引、检索、组装上下文、生成——并给出「排查从右往左、修复从左往右」这条口径：从右往左是因为你最先看到的是生成结果，从左往右是因为左边的错会被右边放大。
补充几个能把环节钉死的症状：答案「半对」多半是切块把一条完整规则切断了；检索结果里混着一眼不相干的东西多半是解析没做干净；模型无视材料用先验知识作答，通常是提示词里少了「只能依据资料回答」；引用编号和内容对不上，那是生成侧漏读或串了行。
最后落到工程做法：这套排查要能重复做，就必须把每次请求的检索结果、进上下文的段落、最终回答一起记下来，否则线上出问题时你根本复现不了。到了要批量做的时候，就得换成一批固定问题加指标，而不是一条条人工看。
可预期的追问：如果检索确实没捞到，改提示词有没有用？答案是没用——材料里没有的东西，再好的指令也只能换一种编法。这句话最能证明你分清了两层。
Key points
Always start by printing the retrieved passages and checking whether the correct answer is present at all.
Split the pipeline into chunking, indexing, retrieval, context assembly and generation; diagnose right to left, fix left to right.
Use symptoms to pin the stage: half-correct answers point at chunking, irrelevant hits at parsing, ignored material at the prompt, mismatched citations at generation.
If retrieval missed the document, prompt changes cannot help; the material simply is not there.
Log retrieved hits, the passages that entered the context, and the final answer together, or production failures are unreproducible.
答题要点
第一步永远是把检索出来的原文打印出来读一遍，判断正确答案在不在里面。
把链路拆成切块、建索引、检索、组装上下文、生成五个环节，排查从右往左、修复从左往右。
用症状钉环节：半对多半是切块问题，混入无关结果多半是解析问题，无视材料多半是提示词缺约束，引用与内容对不上是生成问题。
检索没捞到时改提示词没有意义，材料里没有的东西模型只能编。
要能重复排查就必须把检索结果、进上下文的段落和最终回答一起记录下来。
Context windows are now in the millions of tokens. Does that make the retrieval step obsolete?上下文窗口已经做到上百万 token 了，检索这一步会被淘汰吗？
Common in ChinaCommon overseasDeep dive#long-context#cost#system-design
How to reason about it · think before answering
This is a position question and it is easy to answer as a binary. The signal is whether you separate what fits technically from what is worth paying for on every request.
Concede the valid half first: bigger windows genuinely absorb part of the use case. For an internal tool over a few dozen stable documents with low traffic, stuffing everything in is the right call and building a retrieval stack would be over-engineering.
Then give three reasons it does not absorb the rest. Cost is the first: context is billed per request, so the same corpus is paid for on every one of ten thousand queries, whereas retrieval only pays for the passages it returns. Prompt caching softens this but does not remove it.
Scale is the second: enterprise corpora run to hundreds of thousands of documents and no window holds them. Attribution and access control are the third: pointing an answer at a specific passage, and showing each user only what they are permitted to see, both have to happen before the material reaches the model.
Add the empirical point: as the supplied material grows, models become less reliable at locating the one relevant fact inside it. More context is not automatically better; fewer and more precise passages often win.
Expected follow-up: does retrieval change shape? Yes. Larger windows allow bigger chunks and more of them, which relieves pressure on reranking and compression. Retrieval gets coarser, it does not disappear.
分析过程 · 先想清楚再作答
这是一道立场题，容易答成非黑即白。判断你有没有做过的地方在于：会不会区分「技术上能不能塞进去」和「工程上该不该每次都塞」，只谈前者的答案一听就是纸上谈兵。
先承认对方有道理的部分：窗口变大确实吃掉了检索的一部分场景。几十篇文档、更新不频繁、调用量不大的内部工具，直接全塞是最省事的选择，为它建一套检索系统是过度设计。
再给三条它吃不掉的理由。第一是成本：材料是按次计费的，同一份材料被问一万次就要付一万次，而检索只付取回的那几段；预填充缓存能缓解但不能消除，缓存也有有效期和命中率。
第二是规模：企业知识库动辄几十万篇，再大的窗口也塞不下，检索是唯一的入口。第三是归因与权限：答案要指回具体某一段，以及不同的人只能看到自己有权访问的材料——这两件事必须在把材料喂给模型之前完成，窗口再大也不解决。
还要补一条经验事实：材料变多之后，模型在长上下文里定位关键信息的稳定性会下降，出现「读了但没读到」。所以「全塞」并不总是等于「效果更好」，很多时候少而准反而更好。
可预期的追问：那检索的形态会不会变？会——窗口变大之后，取回的块可以更大、条数可以更多，重排与压缩的压力变小，检索从「精挑几句」变成「粗筛一批」。趋势是检索的粒度变粗，不是检索消失。
Key points
Separate whether it fits from whether it is worth paying for on every request.
Small, stable, low-traffic corpora can legitimately be stuffed whole; building retrieval for them is over-engineering.
Three reasons retrieval survives: per-request cost, corpora too large for any window, and attribution plus access control that must happen before the model sees the material.
More supplied context reduces the reliability of locating a single fact, so stuffing everything is not automatically better.
The trend is coarser retrieval — bigger chunks, more of them, less reranking pressure — not the removal of retrieval.
答题要点
先区分「能不能塞进去」和「该不该每次都塞」，前者是技术问题，后者是成本问题。
小规模、低频、少变的语料确实可以直接全塞，为它建检索系统是过度设计。
检索不会被淘汰的三个理由：按次计费的成本、几十万篇塞不下的规模、必须在喂给模型之前完成的归因与权限过滤。
材料越多，模型定位关键信息的稳定性越差，全塞不等于效果更好。
趋势是检索粒度变粗——块更大、条数更多、重排压力变小，而不是检索消失。

D2 Embeddings and Vector Search: Similarity, Dimensionality, and Model Choice; Storing Text in pgvector

When are cosine similarity and inner product equivalent? What goes wrong if you rank by inner product on vectors that are not normalised?余弦相似度和内积什么时候等价？如果向量没有归一化，用内积排序会出什么问题？
Common in ChinaCommon overseasBasic#embeddings#similarity#normalisation
How to reason about it · think before answering
This starts as a giveaway, but the second half is where candidates separate. Many can say 'they are equivalent after normalisation'; few can describe what breaks without it.
State the definition: cosine similarity is the inner product divided by the product of the two magnitudes. When both magnitudes are 1, the divisor is 1 and cosine reduces to the inner product. That is the whole argument.
Then the failure mode: an un-normalised inner product mixes 'how aligned' with 'how long'. Longer texts tend to produce larger-magnitude vectors, so ranking drifts systematically toward long documents, the same bias BM25's b parameter exists to counter.
Stress that this bug is silent. Nothing throws, results still look plausible, and only an offline evaluation reveals the drift. Hence the engineering rule: normalise once at the embedding boundary, never at each call site.
Add Euclidean distance for completeness: on normalised vectors, squared L2 equals 2 minus twice the inner product, a monotone function of cosine distance, so all three metrics produce the same ranking.
Expected follow-up: which pgvector operator should you use? Since the vectors are normalised, `<=>` and `<#>` rank identically; prefer `<=>` for readability and because it stays correct if someone later forgets to normalise.
分析过程 · 先想清楚再作答
这题是送分题，但区分度藏在后半句。只答「归一化之后两者等价」的人很多，面试官真正想听的是「没归一化会怎么坏」，因为那是线上真的会发生的事。
先把定义摆出来：余弦相似度等于内积除以两个向量模长的乘积。模长都是 1 时除数就是 1，所以余弦相似度就是内积——这一句话就是等价的全部理由，不需要额外的假设。
再说没归一化的后果：内积里混着「方向有多一致」和「向量有多长」两层信息。文本越长，模型输出的向量模长往往越大，于是排序会系统性地偏向长文档——这跟 BM25 里 b 参数要压的是同一个毛病，只是换了个地方冒出来。
点出这类 bug 的性质：它不报错。程序照常跑、结果照常出，只是名次悄悄偏了，你要跑一轮离线评估才可能发现。所以工程上的做法是在 embedding 的出口统一归一化一次，而不是靠每个调用点自觉。
补一句欧氏距离：向量都归一化之后，欧氏距离的平方等于 2 减去 2 倍内积，也就是余弦距离的单调函数，三种距离排出来的名次完全一致。这一句能说明你理解的是关系而不是三条并列的规则。
可预期的追问：那 pgvector 里该用哪个运算符？答案是既然已经归一化，`<=>`（余弦距离）和 `<#>`（负内积）名次一样，选 `<=>` 的理由是可读性和「就算哪天有人漏了归一化也不至于错」。
Key points
Cosine equals inner product divided by both magnitudes; with unit magnitudes the divisor is 1, so they coincide.
Without normalisation the inner product carries magnitude, and longer documents usually have larger magnitudes, biasing the ranking.
The failure is silent, so normalise once at the embedding boundary and verify with offline evaluation.
On normalised vectors L2 and cosine are monotonically related, so all operators rank the same.
In pgvector the operators are `<->` for L2, `<#>` for negative inner product and `<=>` for cosine distance.
答题要点
余弦相似度 = 内积 / 两个模长之积，模长为 1 时除数为 1，两者等价。
没归一化时内积混入模长信息，长文档的向量模长普遍更大，排序会系统性偏向长文档。
这类错误不报错，只能靠离线评估发现，所以要在 embed 出口统一归一化。
归一化之后欧氏距离与余弦距离互为单调函数，三种运算符名次一致。
pgvector 里对应 `<->`（L2）、`<#>`（负内积）、`<=>`（余弦距离）三个运算符。
What do you lose when you cut embedding dimensions from 1536 to 512, and when is that loss acceptable?把 embedding 维度从 1536 降到 512，你会损失什么？什么场景下这个损失可以接受？
Common in ChinaCommon overseasIntermediate#embeddings#dimensions#cost
How to reason about it · think before answering
This is a cost-modelling question. 'Lower dimensions are cheaper but less accurate' earns nothing; the interviewer wants a cost model and a decision order.
Lay out three costs: storage and memory (vector count times dimensions times bytes per dimension, which an ANN index must hold in RAM), query latency (roughly linear in dimensions), and retrieval quality, whose returns diminish sharply at the high end.
Explain why truncation works at all: models trained with Matryoshka representations pack the most important information into the leading dimensions, so truncating and re-normalising keeps the vector usable. It is still lossy, and how lossy is an empirical question on your own data.
Give the decision order: derive a dimension ceiling from your memory budget, then step down two or three notches and measure the metric drop. Choosing the largest model first and optimising cost later usually means redoing the work.
Name the acceptable cases: large corpora of low individual value, pipelines where a reranker recovers some of the loss, and latency-critical online paths. Be conservative where a single miss is expensive, such as legal or clinical retrieval.
Expected follow-up: can different documents use different dimensions? No. Every vector in an index must share one dimension, so changing it means rebuilding the whole index, the same migration cost as changing models.
分析过程 · 先想清楚再作答
这题考的是你会不会算账。只说「维度越低越省、精度越低」的答案没有区分度，面试官在等一个具体的成本模型和一个决策顺序。
先把三笔账列出来：存储与内存（向量数量乘维度乘每维字节数，近似最近邻索引要把它放进内存，所以基本等于机器预算）、检索延迟（每次比较就是一轮乘加，维度大致线性影响耗时）、检索质量（收益递减，低维段每加一档提升明显，高维段加倍只换来很小的改善）。
再说清降维为什么可行：主流模型用套娃式表示训练，重要信息压在靠前的维度上，所以直接截短再归一化仍然可用，这不是另训了一个小模型。截短必然有损失，损失多少只能在自己的数据上跑评估才知道。
给出决策顺序：先按存储与内存预算倒推一个维度上限，再从上限往下试两三档，看指标掉多少，掉得能接受就用低的。反过来「先选最高维再想办法省钱」基本都会返工。
点出可接受的典型场景：库很大而单条价值不高（比如日志、工单）、召回之后还有重排兜底（重排能把粗排的损失补回来一部分）、或者对延迟极敏感的在线场景。反过来法务、医疗这类一条都不能漏的场景就要谨慎。
可预期的追问：能不能不同文档用不同维度？不能——同一个索引里所有向量必须同维，改维度等于全库重建，这跟换模型是同一类迁移成本。
Key points
Three costs: storage and index memory, query latency, and retrieval quality; the first two scale with dimensions, the third has diminishing returns.
Matryoshka representations make truncation viable, but it is lossy and the loss must be measured on your own data.
Decide by deriving a ceiling from the memory budget, then stepping down and measuring.
Truncation pays off for large corpora, low-value items, latency-sensitive paths, and pipelines with a reranker.
All vectors in one index share a dimension, so changing it forces a full rebuild.
答题要点
三笔账：存储与索引内存、检索延迟、检索质量，前两笔随维度近似线性，第三笔收益递减。
套娃式表示让截短再归一化仍然可用，但一定有损失，损失多少要在自己的数据上评估。
决策顺序是先按内存预算定上限，再往下试档位看指标掉多少。
库大、单条价值低、后面还有重排兜底、对延迟敏感的场景，降维划算。
同一索引里维度必须一致，改维度等于全库重建。
Why do some embedding models require different prefixes for queries and documents? What happens if you skip them, and how would you catch it before shipping?为什么有些 embedding 模型要求查询和文档加不同的前缀？不加会怎样，你怎么在上线前发现这个问题？
Common in ChinaCommon overseasIntermediate#embeddings#model-selection#evaluation
How to reason about it · think before answering
The core of this question is silent failure. Reciting 'e5 needs query: and passage: prefixes' is the baseline; explaining why nothing errors out and how you would catch it is what shows experience.
The reason: these models are trained on pairs, short questions on one side and longer passages on the other, two genuinely different distributions. The prefix is a role marker learned during training. Omit it at inference and you are off-distribution.
The consequence: the model still returns vectors, distances still compute, results still have an order, quality just degrades. Nothing throws, exactly like forgetting to normalise.
How to catch it: run a small labelled question set against the same corpus twice, with and without prefixes, and compare hit rate. That is the evaluation gate built on day 8, and catching silent regressions is precisely what it is for.
Mention the sneakier variant: prefixing at index time but not at query time, or using the same prefix on both sides. Everything sits in one coordinate space and looks healthier, yet the query-document alignment is wrong and the loss is just as invisible. Encapsulate prefixes in the embedding call convention rather than hand-writing them everywhere.
Expected follow-up: do OpenAI models need prefixes? No, they are not in that family, so this is not a universal rule but a per-model detail you re-check on the model card every time you switch.
分析过程 · 先想清楚再作答
这题的题眼是「静默失效」。会背「e5 要加 query 和 passage 前缀」只能拿基础分，能说清它为什么不报错、以及怎么在上线前抓住它，才是做过的人。
先讲原因：这一族模型是拿成对数据训练的，一侧是短问句、一侧是长段落，两者的分布本来就不一样。前缀是训练时给模型的角色标记，告诉它这一段该按查询编码还是按文档编码。推理时不给，模型就落在了训练分布之外。
再讲后果的性质：不加前缀模型照样输出向量、照样能算距离、名次照样有先后，只是整体质量下滑。**没有任何报错**——这跟忘了归一化是同一类问题：错误不会自己浮出来。
怎么发现：唯一可靠的办法是一小份标注问题集，用同一批文档跑两遍（加前缀与不加前缀），比命中率。这就是第 8 天要做的评估闸门，它的价值恰恰在于抓这类静默错误。上线前跑一遍，比读十遍文档管用。
补一个更容易踩的变体：**建库时加了前缀、查询时忘了加**，或者两边加成同一个前缀。这种情况下所有向量都在同一个坐标系里，看起来更「正常」，但查询与文档的对齐关系是错的，掉分同样查不出来。所以前缀应该封装在 embed 的调用约定里，而不是散在各处手拼。
可预期的追问：OpenAI 的模型要不要加前缀？不需要——它不属于这一族。所以这不是一条普遍规则，而是**每换一个模型都要重新读模型卡片确认**的事。
Key points
These models are trained on question-passage pairs; the prefix marks which role a text plays, and omitting it puts you off-distribution.
Skipping prefixes never errors, it only degrades quality, so the failure is silent.
The reliable detection is an A/B run over a small labelled question set, comparing hit rate.
A subtler bug is mismatched or identical prefixes on both sides, which looks healthier but misaligns queries and documents.
Keep prefixes inside the embedding call convention, and re-read the model card whenever you switch models.
答题要点
这类模型用问句与段落的成对数据训练，前缀是区分两种角色的标记，缺了就落在训练分布之外。
不加前缀不会报错，只会整体掉分，属于静默失效。
唯一可靠的发现方式是拿一份标注问题集跑 A/B 对比命中率。
更隐蔽的错法是两边前缀不一致或用了同一个前缀，看起来更正常但对齐是错的。
前缀应封装在 embed 的调用约定里；换模型必须重读模型卡片，它不是普遍规则。
Can vector search fully replace keyword search? Give a query where vectors are bound to fail, and say how you would fix it.向量检索能完全取代关键词检索吗？举一个向量必然失手的查询，并说说你会怎么补。
Common in ChinaCommon overseasIntermediate#hybrid-search#embeddings#retrieval-failure
How to reason about it · think before answering
This is a stance question where the stance matters less than the counter-example. Without a concrete, reproducible failing query, the rest of the answer reads as theory.
Enumerate the failure classes up front: error and status codes, version numbers and SKUs, names and employee IDs, order or document identifiers, and negation. The first four share one property: their value lies in exact literal identity, which embeddings deliberately blur into semantic neighbourhoods.
Give a reproducible example: ask whether rate limiting returns 429. BM25 lands on the API document that literally contains 429, while vector search may rank a topically similar product manual that never mentions the code.
Call out negation separately: 'supports PDF export' and 'does not support PDF export' sit almost on top of each other because they discuss the same thing. Vectors cannot carry that distinction; the generation step reading the source has to.
The fix: run both retrievers and fuse the rankings, BM25 on the lexical side and nearest neighbour on the vector side, combined with reciprocal rank fusion. That is hybrid search, covered on day 9. Fusion helps precisely because the two systems fail on different queries.
Expected follow-up: could you drop the keyword path and rewrite queries instead? Rewriting helps with vocabulary mismatch, but it cannot rescue exact identifiers, since there is no paraphrase of 429.
分析过程 · 先想清楚再作答
这题是典型的「立场题」，答「能」或「不能」都不重要，重要的是你能不能举出一个具体到能复现的反例。举不出例子，前面说得再漂亮也会被判成没做过。
先给失手的类型，一次给全：错误码与状态码（429、E1032）、版本号与型号（v2.3.1、X20 Pro）、人名与工号、订单号与文档编号、以及否定表达。前四类的共同点是**这些词的价值在于字面唯一，而向量只保留语义邻近**，模型会把 429 和「限流」「超时」这些话题相近的东西编到一起，反而把真正写着 429 的那篇挤下去。
拿一个能复现的例子说：问「限流超了返回 429 吗」，BM25 稳稳命中写着 429 的接口文档，向量却可能把话题相近但没提 429 的产品手册排在前面。这个现象在本课第 2 天的实验里就能亲眼看到。
否定表达要单独强调：「支持导出 PDF」和「不支持导出 PDF」在向量空间里几乎重合，因为它们谈的是同一件事。指望向量区分肯定与否定一定翻车，这一层要靠生成侧读原文来判断。
怎么补：两路并行跑再融合，关键词一路用 BM25、向量一路用最近邻，用倒数排名融合把两个名次合成一个。这就是混合检索，本课第 9 天展开。要点是**两套的错法不一样**，所以合起来才有增益——如果两套错在同一批查询上，融合是白做的。
可预期的追问：那关键词一路能不能扔掉、改成让模型改写查询？可以缓解一部分（第 10 天的查询改写），但改写救不了字面唯一的标识符——你没法把 429 改写成别的说法。
Key points
No: codes, version numbers, names and IDs matter as exact literals, which embeddings blur into neighbourhoods.
Concrete example: asking whether rate limiting returns 429, where BM25 hits the document containing 429 and vectors surface a topically similar one that never mentions it.
Negation is a second failure class, since affirmative and negative statements sit almost on top of each other.
The remedy is hybrid retrieval: run both paths and merge with reciprocal rank fusion.
Fusion pays off because the two paths fail differently; query rewriting helps vocabulary mismatch but not exact identifiers.
答题要点
不能取代：错误码、版本号、人名、单号这类词的价值在于字面唯一，向量只保留语义邻近。
具体反例：问「限流超了返回 429 吗」，BM25 命中写着 429 的文档，向量把话题相近却没提 429 的文档排前面。
否定表达是另一类失手：肯定句与否定句在向量空间里几乎重合。
补法是混合检索：两路并行再用倒数排名融合合并名次。
融合有增益的前提是两套的错法不同；查询改写能缓解词汇不匹配，但救不了字面唯一的标识符。

D3 Getting Documents In: Parsing PDF and HTML, Tables and Scans, Cleaning Rules, and Metadata You Must Keep

The text extracted from a PDF comes out in the wrong order. How do you diagnose and fix it?一份 PDF 解析出来的文字顺序是乱的，你会怎么排查和修复？
Common in ChinaCommon overseasIntermediate#pdf-parsing#ingestion#data-quality
How to reason about it · think before answering
This checks whether you have actually parsed a PDF yourself. The first sentence is the differentiator: a PDF has no reading order at all, only drawing instructions with coordinates.
Start with the diagnostic step: dump the extracted fragments together with page, x, y and font size instead of looking at the concatenated string. The cause is always in the coordinates.
Then classify the symptom. Lines alternating between left and right means multi-column layout was not detected. Fragments with y jumping backwards means the content stream was written in drawing order. Clean text sprinkled with a repeated short line is not disorder at all, it is a header or footer that was never stripped.
Match the fix to the symptom. For columns, rebuild the order: sort the left edges of the fragments on each page, take the widest gap as the column boundary, then sort by column, then y descending, then x ascending. For headers and footers, cut fixed bands at the top and bottom and print how many fragments you dropped so you can confirm you did not cut into the body.
Add the production-grade part: the fix needs a regression signal, not an eyeball check. Compute an out-of-order score by walking the sorted fragments and counting backward jumps within a column plus right-to-left column jumps. It needs no ground truth, so it can run on every ingest.
Expected follow-up: what if column detection is wrong? Keep the detector conservative, treating a narrow gap or a lopsided split as single column, and make sure the assertion still fires when a two-column page is misread as one. Missing a fix is better than silently corrupting the order.
分析过程 · 先想清楚再作答
这题在考你有没有真的动手解析过 PDF。区分度在第一句：能不能说出「PDF 里根本没有阅读顺序」这个前提。答不出这句的人，后面只会说「换个库试试」。
先给排查顺序：把抽出来的文本片段连同页码、坐标、字号一起打印出来，别只看拼好的字符串。乱序的原因几乎都藏在坐标里，看纯文本永远看不出来。
然后按现象分三类。左右两栏一行一行地交替，是多栏没识别；同一段话被拆成很多短片段且 y 值有回跳，是内容流按绘制顺序写的；文字整体没问题但夹着重复出现的短句，那不是乱序，是页眉页脚没剔。
修法对应着来：多栏就重建阅读顺序——把每页文字块的左边界排序找最大空隙当分栏线，再按「栏号、y 从大到小、x 从小到大」重排；页眉页脚按固定的 y 值带切掉，并打印剔除条数确认没误伤。
补一条能证明你在生产里干过的话：修完要有可回归的判据，不能靠肉眼。用乱序疑似度——顺着排好的顺序走一遍，统计「同栏内往回跳」和「从右栏跳回左栏」的比例，它不需要标准答案，可以挂进流水线天天跑。
可预期的追问：多栏识别错了怎么办？回答分两头——把分栏判定做保守（空隙不够宽、或者一侧内容占比太低就按单栏处理），并且让断言在双栏被误判成单栏时同样会报警，宁可漏修也不要悄悄改错。
Key points
State the premise: a PDF stores only drawing instructions, so paragraphs and reading order are inferred, not read.
Debug by dumping fragments with page, coordinates and font size; plain text hides the cause.
Three common causes: undetected multi-column layout, content stream written in drawing order, and headers or footers left in.
Fix columns by finding the widest gap between left edges and sorting by column, then y descending, then x ascending.
Add a ground-truth-free regression metric such as an out-of-order score so the fix stays fixed.
答题要点
前提先说清：PDF 只存「在某页某坐标画某段文字」，段落和阅读顺序都是解析时推出来的。
排查时把片段连同页码、坐标、字号一起打印，纯文本看不出乱序的原因。
三种典型成因：多栏没识别、内容流按绘制顺序写、页眉页脚没剔除。
多栏的修法是找最大 x 空隙定分栏线，再按「栏号、y 降序、x 升序」重排。
修完要有不依赖标准答案的回归指标，比如乱序疑似度，能挂进摄取流水线。
Which metadata should a document parsing stage preserve, and which downstream feature breaks if you drop each one?文档解析阶段应该保留哪些元数据？少了其中某一项会在哪个环节出问题？
Common in ChinaCommon overseasIntermediate#metadata#ingestion#access-control
How to reason about it · think before answering
The trap here is answering with a bare list. The differentiator is pairing every field with a concrete downstream feature. Listing eight fields without naming who consumes them shows you never designed one.
Give the selection rule first: can this be recovered from the original file later? If not, it must be captured at parse time. Formatting and whitespace can be dropped because the original still has them.
Then map fields to consumers: a stable chunk id makes citations verifiable, a heading path tells the user which section a sentence came from and enables structure-aware chunking, page numbers make citations land on the right page, an access-control label enables filtering inside retrieval, an updated-at date resolves conflicting sources, and a content hash enables incremental sync.
Take two of them all the way to cost. Without the access label you must re-parse the whole corpus when access control lands, and worse, people work around it by filtering at generation time, which means the content already reached the context and the leak already happened.
Without a content hash, every sync is a full rebuild: re-parse, re-chunk, re-embed. For a few thousand documents synced daily, the embedding bill alone settles the argument.
Expected follow-up: what about a field you are unsure of? Be conservative. Storage is the cheapest part of the pipeline, and adding a field costs far less than re-running a full parse.
分析过程 · 先想清楚再作答
这题最容易答成列清单。区分度不在你能列出几个字段，而在能不能给每个字段配一个具体的下游功能——列了八个字段却说不出谁在用，等于没设计过。
用一条判据把字段选出来：删掉之后还能不能从原件重新恢复。不能恢复的，解析时就必须留；能恢复的（比如格式、空白）可以放心丢。
然后一一对应地说：块编号支撑可验证的引用，没有它引用就只能靠模型自觉；标题路径支撑「这句话出自哪一节」和按结构切块；页码支撑引用精确到页；权限标签支撑检索层过滤；更新时间支撑材料冲突时的取舍；内容指纹支撑增量同步。
挑两个讲透代价。权限标签少了，等到要做访问控制时只能全量重新解析一遍；更糟的是有人会图省事在生成阶段过滤，那等于内容已经进了上下文，泄露已经发生。
内容指纹少了，每次同步都是全量重建：重新解析、重新切块、重新向量化。一份几千篇的知识库每天重算一次，光 embedding 的账单就够说服任何人。
可预期的追问：字段拿不准要不要留怎么办？答保守——存储是整条链路上最便宜的一环，加一个字段的代价远小于重跑一次全量解析。
Key points
The rule is recoverability: if it cannot be recovered from the original later, capture it at parse time.
Chunk ids back verifiable citations, heading paths back localisation and structure-aware chunking, page numbers make citations land precisely.
Access-control labels must be attached during parsing, otherwise enabling ACL means re-parsing everything, and teams end up filtering at generation time where the leak has already occurred.
Updated-at lets you present conflicting sources side by side; a content hash enables incremental sync instead of full rebuilds.
When unsure, keep the field: storage is far cheaper than a full re-parse.
答题要点
判据是「删了还能不能从原件恢复」，不能恢复的必须在解析时留下。
块编号服务于可验证的引用，标题路径服务于定位与按结构切块，页码服务于引用精确到页。
权限标签必须在解析时打上，否则做访问控制时要全量重解析，且容易被错误地放到生成阶段过滤。
更新时间用于材料冲突时并列两种说法，内容指纹用于增量同步，少了它每次都要全量重建。
拿不准就保守保留：加一个字段的成本远低于重跑一次全量解析。
OCR output from scanned documents carries a non-trivial error rate. How does that noise propagate into retrieval and generation, and how do you mitigate it?扫描件走光学字符识别之后错字率不低，这些噪声会怎样影响检索和生成？怎么缓解？
Common in ChinaCommon overseasDeep dive#ocr#data-quality#hybrid-search
How to reason about it · think before answering
This tests whether you can trace propagation rather than recite that OCR makes mistakes. The differentiator is separating how retrieval fails from how generation fails, because the two failure modes are entirely different.
Retrieval first. Chinese OCR errors are mostly visually similar characters. Keyword search is literal, so one wrong character makes the term unmatchable, and bigram tokenisation makes it worse because a single wrong character corrupts two adjacent tokens. Recall drops quietly and nothing raises an error.
Generation second. The model usually reads through minor noise, but when the corrupted token is a key entity such as a name, a model number, an amount or a date, it answers confidently with the wrong value. Citation checking degrades too: verifying against a source that is itself wrong proves nothing.
Mitigate in three layers. At ingest, use an empty-text assertion to decide whether the PDF even needs OCR, and keep a link to the original image so a human can verify.
At retrieval, hybrid search absorbs some of the damage because dense retrieval is less sensitive to a single wrong character than literal matching. At generation, mark low-confidence pages so the answer can state that the source came from a scan and may contain recognition errors.
Expected follow-up: can you auto-correct? Yes, but carefully. Dictionary or model based post-processing fixes some errors and breaks correct proper nouns. Restrict correction to low-confidence spans and keep the raw text so you can fall back.
分析过程 · 先想清楚再作答
这题考的是你会不会顺着链条推传导，而不是背「OCR 会有错字」这句废话。判据是有没有分别说清「检索侧怎么错」和「生成侧怎么错」——它们的失效方式完全不同。
先说检索侧。中文 OCR 的错主要是形近字，「已」认成「己」、「板」认成「版」。关键词检索是字面匹配，一个字错了这个词就查不到；更隐蔽的是二元组分词会连带毁掉相邻两个词元，一个错字影响的其实是两处。这一路的表现是召回悄悄掉下去，而且不报错。
再说生成侧。错字进了上下文，模型往往能读懂大意，但一旦是关键实体（人名、型号、金额、日期）出错，它会照着错的答，而且答得很自信。更麻烦的是引用校验也会跟着失效——原文本身就是错的，校验通过了也没意义。
缓解按三层说。入口层：先用空文本比例这类断言判断这份 PDF 有没有文本层，有就别走 OCR；真要走，保留原图链接以便人工复核。
检索层：靠混合检索兜底，向量一路对个别错字不敏感，能补上关键词一路的失手，这是 D9 那套东西在这里的具体价值。生成层：把低置信度的页面标出来，让模型在引用它们时明确提示「该材料来自扫描件，可能有识别误差」。
可预期的追问：能不能自动纠错？可以但要克制——用词典或模型做后处理会修好一批，也会「修」坏一批原本正确的专有名词。稳妥的做法是只对置信度低的片段做纠错，并且保留原文以便回退。
Key points
Retrieval: visually similar characters break literal matching, and bigram tokenisation lets one bad character corrupt two tokens, so recall drops silently.
Generation: the model reads through general noise but confidently repeats corrupted entities, and citation verification against a corrupted source proves nothing.
At ingest: check for a text layer before running OCR at all, and keep the source image for human verification.
At retrieval: hybrid search helps because dense retrieval tolerates a single wrong character better than literal matching.
At generation: flag low-confidence sources in the answer, and restrict auto-correction to low-confidence spans while keeping the raw text.
答题要点
检索侧：形近字让字面匹配直接查不到，二元组分词还会让一个错字毁掉相邻两个词元，表现是召回悄悄下降且不报错。
生成侧：模型能读懂大意，但关键实体出错时会自信地答错，引用校验也失去意义。
入口层缓解：先判断有没有文本层再决定要不要 OCR，并保留原图链接供人工复核。
检索层缓解：混合检索里的向量一路对个别错字不敏感，能兜住关键词一路的失手。
生成层缓解：标出低置信度来源，让回答显式提示可能存在识别误差；自动纠错只对低置信片段做并保留原文。
Why is parsing quality the ceiling on retrieval quality? Walk through one concrete chain of propagation.为什么说解析质量决定了检索质量的上限？举一个具体的传导链条。
Common in ChinaCommon overseasBasic#ingestion#data-quality#failure-analysis
How to reason about it · think before answering
This is a giveaway question that many people answer with a slogan. The only test is whether you produce a chain that lands on a concrete symptom instead of repeating garbage in, garbage out.
Place it first: parsing sits before chunking, indexing, retrieval, context assembly and generation. Its errors are amplified by every later stage, and none of those stages can detect the problem because each is faithfully processing text that is already wrong.
Give the chain: a pricing table in a PDF loses one column separator and comes out with cells shifted. Chunking splits on those wrong boundaries, so a plan name ends up next to the neighbouring column value. The index records the wrong term pairing. A user asks about that plan's storage quota, the corrupted chunk scores highest, and the model, faithfully answering only from the provided material, returns a wrong answer carrying a correct-looking citation.
Name the nastiest part: nothing on that chain raises an error, and the answer even comes with a source, so it looks more trustworthy than usual. Parsing errors cannot be caught after the fact, only by assertions at ingest.
Explain the word ceiling: every later optimisation, dense retrieval, hybrid search, reranking, query rewriting, improves how well you pick from the candidates. If the material itself is wrong, picking better still returns something wrong, so parsing caps all of them.
Expected follow-up: how do you prove parsing is at fault? Reuse the habit from day one. Diagnose right to left and print the retrieved passages verbatim. If the source text is already scrambled, there is no point looking at the generation side.
分析过程 · 先想清楚再作答
这是一道送分题，但很多人答成口号。判据只有一个：有没有给出一条能落到具体现象上的链条，而不是重复一遍「垃圾进垃圾出」。
先说清位置：解析在切块、建索引、检索、组装、生成这五环之前，是第零环。它的错误会被后面每一环放大，而且后面每一环都无法察觉——它们只是在忠实地处理一段已经错了的文字。
给一条具体链条：一张套餐配额表在 PDF 里丢了一列分隔符，抽出来串了行；切块照着错误的边界切，「专业版」和隔壁那一栏的值被切进同一块；索引把错误的词对记进倒排表；用户问「专业版存储配额多少」，这一块分数很高被排到第一；模型只依据给定材料回答，于是给出一个错误但带着正确引用编号的答案。
点破最要命的一句：这条链上没有任何一环会报错，回答甚至是带出处的，看起来比平时更可信。所以解析的错误不能靠事后发现，只能靠入口处的断言拦。
反过来说明「上限」二字：后面所有优化——向量、混合检索、重排、查询改写——优化的都是「从候选里挑得更准」。材料本身错了，挑得再准也是错的，所以它们的天花板由解析封死。
可预期的追问：那怎么证明是解析的锅？答案接回 D1 那条习惯——排查从右往左看，把检索出来的原文打印出来自己读一遍，如果原文本身就是串行的，那就不用再往生成侧查了。
Key points
Parsing is stage zero, before the five-stage pipeline; its errors are amplified downstream and invisible to every later stage.
Concrete chain: a shifted table, chunking on wrong boundaries, wrong term pairs in the index, that chunk ranked first, and a wrong answer delivered with a citation.
The dangerous part is that nothing errors out and the answer carries a source, so it looks more credible than usual.
Later techniques only improve selection from candidates; if the material is wrong, better selection still returns something wrong.
Diagnose right to left: print the retrieved passages first, and if the source text is already broken, stop looking at the generation side.
答题要点
解析是五个环节之前的第零环，它的错误会被后面每一环放大，而后面每一环都察觉不到。
具体链条：表格串行 → 切块按错误边界切 → 倒排表记进错误词对 → 检索把它排第一 → 模型据此给出带引用的错误答案。
最危险的是全程零报错，且答案带着出处，看起来比平时更可信。
后面所有优化解决的是「挑得更准」，材料本身错了就都无效，所以上限由解析封死。
定位方法是排查从右往左：先把检索到的原文打印出来读一遍，原文错了就不必再查生成侧。

D4 Chunking Strategies: Five Approaches — Fixed, Recursive, Structure-Based, Parent-Child, and Semantic — and Choosing by Evaluation, Not Intuition

How do you decide on chunk size? Name two metrics you would look at, and one counterexample.你怎么决定切块大小？说出你会看的两个指标和一个反例。
Common in ChinaCommon overseasIntermediate#chunking#evaluation
How to reason about it · think before answering
The question is about method, not about a number. Answering with a specific default (512 tokens, 1000 characters) already loses it — the interviewer wants to hear that you have a procedure.
State the tension first: large chunks dilute the signal and cost context; small chunks lose the surrounding meaning so the model cannot use them. The two metrics you name should map onto those two failure modes.
Metric one is retrieval-side hit rate: did a document that actually answers the question make it into the context. Metric two is generation-side usability, cheaply proxied by the fraction of chunks that end mid-sentence, and more seriously by faithfulness and whether citations resolve.
Add the point that separates candidates: both metrics must be compared under the same token budget, never under a fixed top-k. With fixed k, bigger chunks simply buy more text and win for the wrong reason.
Make the counterexample concrete: raising chunk size from 400 to 1200 characters can lift hit rate purely because whole short documents now fit in one chunk, which means retrieval stopped doing anything and you are back to stuffing full documents. The metric improved while the system got worse.
Expect the follow-up: where do you start on day one. Pick the strategy from the document type first (structural splitting whenever headings exist), start around 300 to 500 characters with 10 to 20 percent overlap, then build a golden set immediately and iterate. A starting point is not a conclusion.
分析过程 · 先想清楚再作答
这题的题眼是「怎么决定」，不是「多大合适」。答一个具体数字（512 token、1000 字符）就已经输了——面试官想看的是你有没有一套定法，而不是你记得住哪个默认值。
先把矛盾摆出来：块大则信噪比低、上下文贵，块小则单块缺语境、模型答不出所以然。切块大小就是在这两头之间找位置，所以两个指标必须分别对应这两头。
第一个指标是检索侧的命中率——答案文档有没有进上下文。第二个是生成侧的可用性，最省事的代理指标是切碎率，也就是有多少块结尾停在半句话上；再往前一步就是忠实度和引用是否可定位。
关键补一句：两个指标必须在**同一个 token 预算**下比，不能按「取前 k 块」比。k 固定时块越大塞进去的字越多，大块切法会赢在买得多而不是切得准上。这一句往往是这道题的区分点。
反例要具体。最好用的一个是：把块从 400 字调到 1200 字，命中率不降反升——但那是因为一整篇短文档被当成一块塞了进去，检索其实什么都没做，等于退化成了全文投喂。指标涨了，系统更差了。
可预期的追问是「那你第一次上手时从哪个数字起步」。答：先按文档类型选切法（有标题层级就按结构切），块长从 300 到 500 字起步、重叠取一到两成，然后立刻建一组标准问题跑评估，用两三轮迭代把它调到位。起步值是起步值，不是结论。
Key points
Choose the strategy from the document type first, then tune length: split on headings whenever the structure survives parsing.
Watch two metrics: retrieval hit rate on one side, mid-sentence break rate (then faithfulness and citation resolvability) on the other.
Compare under an equal token budget, never a fixed top-k, or larger chunks win by buying more text.
Counterexample: hit rate rises after enlarging chunks because whole documents now fit in one chunk and retrieval has effectively stopped working.
Start near 300 to 500 characters with 10 to 20 percent overlap, then iterate against a fixed question set instead of guessing.
答题要点
先按文档类型选切法，再调长度：有标题层级就按结构切，没有结构才谈固定长度或语义。
看两个指标：检索侧的命中率，生成侧的切碎率（进一步是忠实度与引用可定位性）。
两个指标必须在同一个 token 预算下比，不能按「取前 k 块」比，否则大块只是买得更多。
反例：块调大后命中率上升，但那是因为整篇被当成一块，检索退化成全文投喂。
起步值 300 到 500 字、重叠一到两成，然后靠一组固定问题迭代，不靠直觉定稿。
What does parent-child chunking buy you, and when does it slow the system down instead?父子切块的收益是什么？它在什么情况下反而会拖慢系统？
Common in ChinaCommon overseasIntermediate#chunking#parent-child
How to reason about it · think before answering
This question checks whether you know that the retrieval unit and the context unit can be two different things. Without that sentence, everything else is recitation.
State the benefit compactly: small chunks go into the index so they are easy to match, and once a child is hit you follow the parent pointer and hand the model the whole section. You stop trading precision against completeness.
Derive the slowdown from the costs. First, the context budget: every new child may drag in an entire parent, so an equal budget holds fewer distinct pieces and result diversity drops.
Second, the write path: two levels to maintain, both recomputed on every document update, and chunk ids become harder to keep stable, which makes incremental sync noticeably more complex.
Third, the condition under which the benefit disappears: when sections are already short, the parent and the child are nearly the same text, so you paid for two indexes and bought nothing. Parent-child suits long sections and deep hierarchies, not already fine-grained knowledge bases.
Expect the follow-up: how is this different from simply using bigger chunks. Bigger chunks put the noise into the index; parent-child puts the noise only into the context. What gets matched stays short and clean.
分析过程 · 先想清楚再作答
这题考的是你有没有意识到「检索单位」和「上下文单位」可以是两个东西。答不出这句话，后面说什么都是复述。
收益一句话说清：小块进索引，信噪比高、容易被找到；命中之后顺着父指针把整节回填给模型，语境完整。精度和完整度这次不用二选一。
拖慢的场景要从代价一条条推。第一条是上下文预算：每命中一个新子块可能拖进来一整个父节，同样的 token 预算装不下几条，检索结果的多样性反而变差。
第二条是写入侧：父子两套都要维护，文档更新时两边都要重算，块 id 的稳定性也更难保证，增量同步的复杂度明显上升。
第三条是收益消失的条件：当文档本身的小节就不长时，父块和子块差不多大，你付了两套索引的钱，什么也没多买到。所以父子切块适合长节、深层级的文档，不适合结构本来就细碎的知识库。
可预期的追问是「那和直接把块切大有什么区别」。答：切大是把噪声一起放进索引，父子是只把噪声放进上下文、不放进索引——被检索的那一段始终是干净的短文本，这是本质区别。
Key points
The core idea is decoupling the retrieval unit from the context unit: small chunks get found, large chunks get understood.
The payoff is precision and completeness at the same time instead of trading one for the other.
Cost one: a single hit can drag in a whole parent, so an equal context budget holds fewer distinct results and diversity suffers.
Cost two: two index levels to maintain and recompute, which makes incremental sync on document updates considerably harder.
It stops paying off when sections are already short, because parent and child are nearly identical and you bought nothing for the extra cost.
答题要点
核心是把检索单位和上下文单位拆开：小块负责被找到，大块负责被读懂。
收益是精度与完整度同时拿到，不用在信噪比和语境之间二选一。
代价一：一次命中可能拖进整个父节，同样的上下文预算装得下的条数变少，结果多样性下降。
代价二：父子两套索引都要维护与重算，文档更新时增量同步的复杂度明显上升。
失效场景：文档小节本来就短时父子块差不多大，多付一套成本却没多买到东西。
What overlap ratio would you use, and what concretely goes wrong when the overlap is too large?重叠区设成块长的百分之多少合适？重叠过大会带来什么具体问题？
Common in ChinaCommon overseasBasic#chunking#overlap
How to reason about it · think before answering
This is a giveaway question, but the marks are in the second half, not the percentage. Stopping at 'usually ten to twenty percent' reads like someone who has never run it.
Say what overlap is patching: fixed-length splitting cuts sentences in half, and overlap guarantees the broken sentence survives intact in at least one of the two neighbours. It is a patch for careless splitting, not an optimisation of its own.
That yields the first conclusion: with structural or recursive splitting the boundaries already land on semantic positions, so the need for overlap drops sharply and can legitimately be zero. The ratio question is meaningless without naming the strategy.
Give three concrete costs. Storage and tokens: at 400-character chunks, moving overlap from 0 to 80 grows total index tokens by roughly fifteen percent, which is storage cost in the vector store and comparison work at query time.
Retrieval redundancy: the more neighbours overlap, the more likely the top results are three versions of the same passage. You think you handed the model three pieces of evidence; you handed it one, three times. Nothing fixes this before reranking.
Citation resolution: when a sentence lives in two chunks, which one does the model cite. Expect the follow-up on deduplication: merge at the result layer using a content fingerprint or longest common substring, not by tweaking the chunker.
分析过程 · 先想清楚再作答
这是一道送分题，但送分点不在那个百分比上，而在后半句。只答「一般一到两成」就停住的人，面试官会认为他没跑过。
先说清重叠在补救什么：固定长度切法会把句子从中间切开，重叠让被切开的那句话至少在相邻两块之一里是完整的。它是给「乱切」打的补丁，不是一个独立的优化。
由此推出第一个结论：如果你用的是按结构切或递归切，边界本来就落在语义位置上，重叠的必要性会大幅下降，甚至可以是零。**重叠比例这个问题的前提是切法**，脱开切法谈比例就是背数字。
过大的代价要说三笔，越具体越好。存储与 token：块长 400、重叠从 0 加到 80，索引 token 会涨一成半左右，这笔钱在向量库是存储费、在检索时是比对量。
检索冗余：相邻块越像，前几名越可能是同一段话的三个版本，你以为给了模型三条证据，其实是一条说了三遍。这一条在重排之前基本无解。
引用定位：同一句话出现在两个块里，模型标出处该标哪一个，这会直接变成引用校验环节要处理的边界情况。可预期的追问就是「那你怎么去重」，答按内容指纹或最长公共子串在结果层合并，而不是在切块层想办法。
Key points
Ten to twenty percent of chunk length is the working range, but that number assumes fixed-length splitting.
With structural or recursive splitting the boundaries are already semantic, so overlap can be small or zero.
Cost one: index tokens and storage grow noticeably; at 400-character chunks, an 80-character overlap adds roughly fifteen percent.
Cost two: neighbouring chunks become near-duplicates, so the top results are several versions of one passage and the evidence diversity is illusory.
Cost three: a sentence spanning two chunks complicates citation attribution and forces result-level deduplication.
答题要点
经验区间是块长的一到两成，但这个数字的前提是你用的是固定长度切法。
按结构或递归切时边界本来就在语义位置上，重叠可以很小甚至为零。
过大代价一：索引 token 与存储明显上涨，块长 400 时重叠加到 80 大约涨一成半。
过大代价二：相邻块高度相似，检索前几名变成同一段话的多个版本，证据多样性是假的。
过大代价三：同一句话跨块出现，引用标注和去重都要额外处理。
Semantic chunking costs considerably more than recursive splitting. How would you prove to your team that the money is well spent?语义切分比递归切分贵不少，你怎么向团队证明这笔钱值得花？
Common in ChinaCommon overseasDeep dive#chunking#evaluation#cost
How to reason about it · think before answering
This looks like a technical question but it tests whether you can run a controlled technical argument. Launching into how semantic chunking works answers a different question.
Step one is to concede that it may well not be worth it. The gain comes from documents that have no usable structure; if your knowledge base is well-formed documents, the authors' heading hierarchy already did the semantic split for free and the money is likely wasted.
Step two is translating 'worth it' into three measurable numbers: how much the metric moved (hit rate on the same golden set under the same token budget), how much latency moved (chunking is offline, but the end-to-end update path changes), and how much it costs (the initial full embedding pass plus recomputation amortised over update frequency).
Step three is the control. Recursive splitting is the baseline, semantic chunking the treatment, and they must share the corpus, the questions, the context budget and the retriever. Change one variable only; a two-variable experiment proves nothing.
Step four is a decision threshold rather than an impression. For example: below three points of hit-rate gain, no; above five points with recomputation inside the monthly budget, yes; in between, roll it out on one document class first. Fix the threshold before you run the numbers, or you will quietly bend it to fit them.
Expect the follow-up: is there a cheaper way to the same gain. Yes — try structural splitting first, since it is free and often nearly as good, and if the structure really is unusable, apply semantic chunking only to the high-value subset rather than the whole corpus.
分析过程 · 先想清楚再作答
这题表面问技术，实际考的是你会不会做一次带对照组的技术论证。上来就讲语义切分原理的人，答的是另一道题。
第一步是先承认它可能不值。语义切分的收益来自「文档没有可用的结构」；如果知识库是结构良好的文档，作者的标题层级已经免费替你做完了语义切分，这时候花的钱大概率打水漂。**先说清适用前提，再谈证明，这一步就把大多数候选人区分开了。**
第二步是把「值不值」翻译成可测的三笔账：指标涨了多少（同一批标准问题、同一个 token 预算下的命中率）、延迟涨了多少（切块是离线的，但更新链路的端到端时间会变）、钱涨了多少（首次全量 embedding 的费用，加上按更新频率折算的重算费用）。只报第一笔的论证不成立。
第三步是设计对照。递归切分是基线，语义切分是实验组，两组必须用同一份语料、同一批问题、同一个上下文预算、同一个检索器，只改切法这一个变量。改两个变量的实验，结论一文不值。
第四步是给决策一个门槛，而不是给一个感想。比如：命中率相对基线提升低于三个百分点就不上；提升超过五个百分点且重算成本在月度预算内就上；中间地带先在一类文档上灰度。**门槛要在跑数字之前定好**，否则你会不自觉地去迁就已经跑出来的结果。
可预期的追问是「有没有更便宜的办法拿到同样的收益」。答有：先试按结构切，它零成本且效果常常接近；结构确实不可用时，再考虑只对高价值的那一部分文档做语义切分，而不是全量上。
Key points
Start with the precondition: the gain comes from documents without usable structure, so on well-formed documents it usually is not worth it.
Translate 'worth it' into three numbers — hit rate, latency, and cost. Reporting only the first is not an argument.
Run a controlled comparison: same corpus, same golden set, same context budget, same retriever, with the splitting strategy as the only variable.
Fix the decision threshold before running the numbers so you cannot bend it to fit the result afterwards.
Try free structural splitting first, and if semantic chunking is genuinely needed, apply it to the high-value subset rather than the entire corpus.
答题要点
先讲适用前提：语义切分的收益来自文档没有可用结构，结构良好的文档上它大概率不值。
把「值不值」翻译成三笔账：命中率涨多少、延迟涨多少、钱涨多少，只报第一笔不算论证。
做对照实验：同语料、同问题集、同上下文预算、同检索器，只改切法一个变量。
决策门槛必须在跑数字之前定好，避免事后迁就结果。
先试零成本的按结构切；确需语义切分时也优先只覆盖高价值文档，而不是全量上。

D5 Vector Indexes and Store Selection: HNSW vs. Inverted File, Quantization to Save Memory, Filtered Queries and Multi-Tenant Isolation

How do you choose between an HNSW index and an IVFFlat index? Give one scenario that forces each choice, and name the parameter you would tune first in each.分层可导航小世界图和倒排文件索引你会怎么选？各说一个必须选它的场景，以及各自最该调的参数。
Common in ChinaCommon overseasIntermediate#vector-index#hnsw#ivfflat
How to reason about it · think before answering
The differentiator is not describing both structures, it is naming the condition that forces one over the other. Saying 'HNSW is faster, IVFFlat is cheaper' is what everyone says.
Describe the structures in one line each: HNSW is a layered neighbour graph you navigate from sparse upper layers down to dense lower ones; IVFFlat clusters vectors into lists and only scans the lists closest to the query.
Map the knobs: HNSW builds with m and ef_construction and queries with ef_search; IVFFlat builds with lists and queries with probes. Tune the query-side knob first, because it needs no rebuild and is the only one you can still move after launch.
Give two forcing scenarios in opposite directions. Minute-level write traffic with tight memory and a short build window forces IVFFlat, since an HNSW graph keeps growing and is expensive to rebuild. A largely static corpus with a hard latency SLA forces HNSW, since it hits the same recall at lower latency.
Add the operational detail people forget: IVFFlat clusters reflect the data at build time, so recall degrades silently as the distribution drifts and you need a scheduled rebuild. HNSW avoids that but its index is often larger than the table.
Expected follow-up: what are the defaults? probes is 1 and ef_search is 40. Volunteer that leaving probes at 1 means scanning a single list, which is the single most common IVFFlat mistake.
分析过程 · 先想清楚再作答
这题的区分度不在能不能背出两种结构，而在你会不会给出触发条件。只说「HNSW 快、IVFFlat 省内存」的人一抓一大把，面试官等的是「什么情况下我必须选另一个」。
先用两句话把结构说清：HNSW 是分层的邻居图，查询从稀疏的上层跳到稠密的下层，逐步逼近；IVFFlat 是先聚类成若干个列表，查询时只在最近的几个列表里扫。一个是图上导航，一个是分区搜索。
再把参数对应上去：HNSW 建图有 m 与 ef_construction，查询有 ef_search；IVFFlat 建索引有 lists，查询有 probes。**先调查询侧参数**，因为它不用重建索引、能逐次查询调整，是唯一一个上线之后还能动的旋钮。
给两个反向的必须场景：数据分钟级高频写入、且内存和建索引窗口都紧张时必须选 IVFFlat，因为 HNSW 的图会持续膨胀、重建代价高；反过来，数据相对静态、查询延迟有硬性 SLA 时必须选 HNSW，因为同等召回下它的延迟更低。
补一条容易被忽略的工程细节：IVFFlat 的聚类是建索引那一刻的数据决定的，数据分布漂移之后召回会悄悄下滑，所以它需要一条定期重建的运维流程；HNSW 没有这个包袱，但它的索引往往比表本身还大。
可预期的追问：probes 和 ef_search 的默认值分别是多少？答 1 和 40，并且要主动说出 IVFFlat 默认 probes = 1 意味着只看一个列表，建完索引不设 probes 基本等于没调过——这是新手最常见的事故。
Key points
HNSW is a layered neighbour graph; IVFFlat clusters first and scans a subset of lists. HNSW favours query quality, IVFFlat favours build cost and memory.
Tune the query-side knob first: ef_search for HNSW, probes for IVFFlat. Neither needs a rebuild.
Heavy write traffic with tight memory and build windows points to IVFFlat; a static corpus with a hard latency SLA points to HNSW.
IVFFlat clusters drift with the data and need scheduled rebuilds; HNSW does not, but its index is often larger than the table.
Know the defaults: probes 1, ef_search 40. Leaving probes at 1 wastes the index.
答题要点
HNSW 是分层邻居图，IVFFlat 是先聚类再局部扫描；前者查询质量优先，后者建索引与内存开销优先。
先调查询侧参数：HNSW 调 ef_search，IVFFlat 调 probes，两者都不需要重建索引。
高频写入、内存与建索引窗口紧张选 IVFFlat；数据相对静态、延迟有硬性要求选 HNSW。
IVFFlat 的聚类会随数据漂移失真，需要定期重建；HNSW 没这个问题但索引常常比表还大。
默认值要记住：probes 是 1、ef_search 是 40，建完索引不调 probes 等于没用上索引的能力。
Why does a vector search with a WHERE clause return fewer results than expected, and what are the fixes and their costs?为什么加了 WHERE 条件的向量检索会漏结果？有哪几种修法，代价分别是什么？
Common in ChinaCommon overseasDeep dive#filtering#iterative-scan#recall
How to reason about it · think before answering
This is the question that separates people who ran a demo from people who ran this in production. The tell is whether you distinguish missing rows from mis-ordered rows.
State the mechanism in one sentence: with approximate indexes, filtering is applied after the index scan. The index first collects ef_search candidates by distance, and only then applies the WHERE clause to that batch.
Do the arithmetic out loud: a condition matching 1% of rows against a default candidate list of 40 leaves well under one row on average. That is why the query looks broken even though the rows exist.
Split the failure into two kinds. Too few rows returned is one; enough rows but the wrong ones ranked first is the other. They have different fixes, and conflating them signals inexperience.
Fix one is iterative scanning, available since pgvector 0.8.0: when too many candidates are filtered out, keep scanning more of the index until enough results are found. Strict ordering keeps exact distance order, relaxed ordering trades slight reordering for better recall, and both cost latency.
Fix two is making the filter apply first: a plain index on the filter column for highly selective conditions, a partial index when there are only a few distinct values, list partitioning when there are many. The costs are losing the approximate speedup, index count exploding per value, and DDL plus operational complexity.
Expected follow-up: how do you pick? Check the returned row count first. Too few means iterative scanning; enough rows with low recall means raising probes or ef_search, or switching to pre-filtering.
分析过程 · 先想清楚再作答
这题是本天的核心，也是最能筛掉「只跑过 demo」的人的一题。题眼在「漏」这个字：能不能说清楚漏的是条数还是排序，直接决定你被归到哪一档。
先讲机制，一句话就够：近似索引的过滤发生在索引扫描之后。索引先按距离取回 ef_search 个候选，然后才拿 WHERE 去筛这一批。条件命中率越低，活下来的越少——命中 1% 的条件配默认的 40 个候选，平均只剩零点几条。
然后把漏召回拆成两类，这是拿分点：一类是**结果条数不够**，十条只给了一两条；另一类是**条数够但排序不对**，十条都在只是排错了。两类的修法完全不同，混为一谈说明没真跑过。
修法一是迭代扫描（pgvector 0.8.0 起）：候选被过滤掉太多时自动回索引里继续扫，直到凑够。它只解决第一类。两种模式的取舍要说清楚——严格顺序保证结果按距离排好，宽松顺序允许略微乱序换更高召回，代价都是延迟明显上升。
修法二是预过滤，即让过滤条件先生效：条件很挑剔时给过滤列建普通索引走精确检索，取值只有少数几个时建部分索引，取值很多时按值做列表分区。代价分别是失去近似索引的加速、索引数量随取值爆炸、以及 DDL 与运维复杂度上升。
可预期的追问：怎么判断该用哪一种？给一条可执行的判据——先看返回条数够不够。不够是第一类，先试迭代扫描；够了但召回低是第二类，只能加大 probes 或 ef_search，或者干脆改成预过滤。
Key points
With approximate indexes the filter runs after the index scan, so a selective condition wipes out most candidates and the query returns too few rows.
There are two failure modes: too few rows, and enough rows in the wrong order. Always check the returned count first.
Iterative scanning fixes only the first. Strict ordering preserves distance order, relaxed ordering gives better recall, and both raise latency noticeably.
Pre-filtering is the alternative: index the filter column for exact search, use a partial index for a few distinct values, partition by value for many. Costs are losing the approximate speedup, index sprawl, and operational complexity.
The second failure mode is only fixed by raising probes or ef_search; iterative scanning does nothing for it.
答题要点
近似索引的过滤发生在索引扫描之后，条件命中率低时候选几乎被筛光，所以返回条数不够。
漏召回分两类：条数不够，和条数够但排序不对。判断顺序永远是先看返回条数。
迭代扫描只修第一类，严格顺序保序、宽松顺序召回更高，代价是延迟明显上升。
预过滤是另一条路：过滤列建索引走精确检索、取值少建部分索引、取值多按值分区，代价依次是失去索引加速、索引数量爆炸、运维复杂度上升。
第二类只能靠加大 probes 或 ef_search，迭代扫描对它完全无效。
If you switch your vectors from full precision to half precision or binary quantisation, how do you verify that recall has not dropped materially?把向量从全精度换成半精度或二值量化，你会用什么方法确认召回没有明显下降？
Common in ChinaCommon overseasIntermediate#quantization#evaluation#recall
How to reason about it · think before answering
The question looks like it is about quantisation, but it is really about whether you know how to evaluate. Answering 'try a few queries and eyeball it' fails immediately.
Pin down ground truth first: it must come from an exhaustive scan with the index disabled. Using index results as ground truth is the classic self-deception, because recall then looks close to 100% no matter what you changed.
Give the procedure: fix a query set of at least a few dozen covering short and long queries across topics, compute ground truth at full precision, rerun with the quantised representation, and report recall at k. Report index size, build time, and median plus p95 latency alongside it, because recall alone is not a decision.
Add the judgement rule: quantisation loss depends on your vector distribution, so published numbers do not transfer. Sparse vectors suffer badly under binary quantisation because only the sign bit survives and zeros collapse together.
Land on something actionable: half precision is usually near lossless and raises the indexable dimension ceiling from 2000 to 4000, so it is a safe first step. Binary quantisation loses real recall and should be used as a cheap first pass, re-ranked with the original vectors over a wider candidate window.
Expected follow-up: how much loss is acceptable? It depends on what comes next. With a re-ranker downstream, a couple of points off first-stage recall is usually invisible; if retrieval feeds the prompt directly, one point means one more unanswerable question per hundred. Tie the threshold to a product metric, not to a number you made up.
分析过程 · 先想清楚再作答
这题表面问量化，实际问的是你会不会做评估。只回答「跑几个问题看看结果对不对」的人会被直接判为没做过——面试官想听的是一套可复现的量法。
先把真值这件事说死：真值必须来自暴力全量比对，也就是把索引关掉、全表算距离取前 k。拿索引结果当真值是最常见的自欺，因为那样量出来的召回永远接近 100%，你会以为量化无损。
然后给流程：固定一批查询（几十条起步，覆盖长短查询和不同主题），先用全精度算出真值，再换量化重跑，计算召回率@k。同时记录三件事——索引大小、建索引耗时、查询延迟的中位数与 p95，只报召回是不够的。
补一条判据：量化损失有多大取决于向量分布，别人的数字不能抄。稀疏向量对二值量化尤其不友好，因为二值化只保留符号位，零和负数会被压成同一个值，信息几乎被抹平。所以换方案必须在自己的数据上重新量一次。
结论要给可操作的建议：半精度通常近乎无损，还能把建索引维度上限从 2000 提到 4000，是默认可以先上的一档；二值量化损失明显，标准用法是拿它粗筛一批候选，再用原始向量在这一小批里精排，粗筛窗口越宽召回补得越多、延迟也越高。
可预期的追问：召回掉了多少算可以接受？答这取决于下游——后面还有重排时，粗排召回掉两三个点通常无感；如果检索结果直接进提示词，掉一个点就意味着每一百次回答里多一次缺材料。要把这个判断挂到业务指标上，而不是拍一个阈值。
Key points
Ground truth must come from an exhaustive scan with indexes disabled; using index output as truth pins recall near 100%.
Run one fixed query set before and after, report recall at k together with index size, build time and latency percentiles.
Quantisation loss depends on your own vector distribution, so measure it on your data instead of quoting benchmarks.
Half precision is usually near lossless and raises the indexable dimension limit from 2000 to 4000, making it a safe default.
Binary quantisation loses real recall; use it as a cheap first pass and re-rank with the original vectors over a wider window.
答题要点
真值必须来自关掉索引的暴力全量比对，拿索引结果当真值会让召回永远接近 100%。
固定一批查询，量化前后跑同一批，报召回率@k，同时报索引大小、建索引耗时和延迟分位数。
量化损失取决于向量分布，别人的数字不能抄，必须在自己的数据上重新量。
半精度通常近乎无损，还能把索引维度上限从 2000 提到 4000，可以作为默认第一档。
二值量化损失明显，正确用法是粗筛加原始向量重排，粗筛窗口越宽召回补得越多、延迟越高。
When should you move your vectors out of PostgreSQL into a dedicated vector database? Give measurable triggers, and also make the case for staying.什么时候应该把向量搬出 PostgreSQL？给出可量化的触发条件，也说说不该搬的理由。
Common in ChinaCommon overseasIntermediate#vector-database#architecture#trade-offs
How to reason about it · think before answering
This tests engineering judgement, not tooling preference. Opening with 'dedicated vector databases are better' invites follow-ups you cannot answer.
State the default position and justify it: keep the first version in PostgreSQL, because transactions, backups, point-in-time recovery, permissions, joins with business tables and the tooling your team already knows all come free. A second datastore adds synchronisation, a consistency surface and an on-call burden that selection documents rarely price in.
Then give four measurable triggers: data volume (the test is whether the index still fits in memory, not the raw row count), write frequency (minute-level streaming updates distort clusters and inflate graphs), filter complexity (arbitrary combinations of a dozen attributes defeat both partial indexes and partitioning), and operational capacity.
Expand on filter complexity, because it is most often the real reason: dedicated vector databases push filtering into the index structure instead of applying it after the scan, which is a mechanical advantage rather than a reputational one.
Volunteer the alternative people skip: many 'vector search is not good enough' problems are actually solved by hybrid retrieval plus re-ranking, not by a new database. Add the keyword path and a re-ranker first, then decide.
Expected follow-up: how would you migrate? Dual-write, compare recall and latency on shadow traffic, shift read traffic gradually, and only then retire the old path. Stop at any step where the metrics regress.
分析过程 · 先想清楚再作答
这题考的是工程判断，不是技术偏好。开口就说「专用向量库更专业」的人会被追问到答不上来；面试官想看的是你有没有把迁移成本算进去。
先给默认立场并给出理由：第一版留在 PostgreSQL，因为事务、备份、时间点恢复、权限、跟业务表 JOIN 和现成的运维工具全是白送的。多一个数据库就多一份同步、一份一致性问题、一份值班负担，这些成本很少被写进选型文档。
然后给四条可量化的触发线：数据量（判据不是行数而是索引还塞不塞得进内存）、写入频率（分钟级流式更新会让聚类失真、让图持续膨胀）、过滤复杂度（十几个属性的任意组合让部分索引和分区都排列组合不过来）、团队运维能力（没人愿意长期照看第二个数据库，前三条再成立也别搬）。
第三条要展开一点，因为它最常是真正的原因：专用向量库把过滤做进了索引结构本身，而不是扫完索引再筛，所以在复杂过滤下天然占优。把这一点说出来，说明你理解的是机制而不是口碑。
还要主动给一条常被忽略的替代路径：很多「向量检索不够用」的问题，真正的解法是混合检索加重排，而不是换数据库。先把关键词一路加回来、把重排接上，再决定要不要搬——顺序搞反了会白搬一次。
可预期的追问：真要搬怎么迁？答分三步——先双写并在影子流量上比对两边的召回与延迟，再把读流量按比例切过去，最后才停掉旧路径。中间任何一步指标不达标就停下，这比一次性切换安全得多。
Key points
Default to staying in PostgreSQL: transactions, backups, recovery, permissions, joins and familiar tooling are free, and a second store adds sync and on-call cost.
Trigger one is data volume, measured by whether the index still fits in memory rather than by row count.
Trigger two is write frequency: minute-level streaming updates distort clusters and inflate graphs.
Trigger three is filter complexity: dedicated stores push filtering into the index structure, a mechanical advantage under complex predicates.
Trigger four cuts the other way: without people to run a second database, do not move even if the first three hold. Often hybrid retrieval plus re-ranking is the real fix.
答题要点
默认留在 PostgreSQL：事务、备份、恢复、权限、JOIN 和现成运维都是白送的，多一个库就多一份同步与值班成本。
触发线一是数据量，判据是索引还塞不塞得进内存，而不是行数本身。
触发线二是写入频率，分钟级流式更新会让聚类失真、让图持续膨胀。
触发线三是过滤复杂度，专用库把过滤做进索引结构，复杂过滤下有机制上的优势。
触发线四反过来看：没有长期运维第二个数据库的人手，前三条成立也不该搬；很多问题的真正解法是混合检索加重排。

D6 The Generation Side: Ordering Context, Labeling Citations, When You Must Refuse to Answer, and Streaming Responses

How do you make sure a model's citations are real rather than fabricated? Describe a scheme that does not rely on the model behaving well.怎么让模型的引用是真的而不是编的？说出一个不依赖模型自觉的方案。
Common in ChinaCommon overseasIntermediate#citation-verification#grounding#hallucination
How to reason about it · think before answering
The phrase to catch is 'not relying on the model behaving well'. Any answer that boils down to 'tell the model to be accurate in the prompt' fails, because the prompt is exactly the part that cannot enforce this.
Split the problem in two. Verifiability requires that a citation be a symbol from a closed set, not free text. So step one is numbering the blocks at assembly time and telling the model it may only cite the numbers it was given. 'According to the storage handbook' cannot be checked, because the title is a string the model can invent.
Step two is post-hoc checking, with two gates. Gate one is existence: you handed out 1 through 5, so an 8 is fabricated, and that is a one-line check. Gate two is substantive overlap, which catches the sneakier case where the number is real but the block says something else. Measure what fraction of the sentence's terms appear in the cited block and reject below a threshold.
Mention the trap in the overlap metric: drop terms that appear in most blocks first, otherwise generic words let any citation pass. It is the same reasoning behind inverse document frequency in BM25.
On failure, feed the specific reason back and regenerate once, not repeatedly. Two fabricated drafts in a row means the material does not support the question, so refuse instead. Also verify against the original chunk text, never against a compressed or rewritten version, otherwise 'verified' says nothing about what the user sees.
Expected follow-up: why not ask the model to self-check? Self-checking shares the generator's bias and has no independent source of truth, whereas number checking is deterministic, essentially free, and reproducible.
分析过程 · 先想清楚再作答
题眼在「不依赖模型自觉」这半句。回答里只要出现「在提示词里强调请确保引用准确」，这题就答砸了——面试官问的正是提示词管不住的那部分。
先把问题拆成两半：引用要能验证，前提是它是一个**闭集里的符号**，不是一段自由文本。所以第一步是组装上下文时给每块材料一个编号，提示词里明确只能引用发出去的编号。让模型写「根据《某某手册》」是没法验证的，标题是它可以随口生成的字符串。
第二步是事后核对，两道闸缺一不可。第一道查编号存在性：发出去的是 1 到 5，出现 8 就一定是编的，一行代码判掉。第二道查实质重合：编号是真的、内容却对不上，这类更隐蔽，要算这句话的词元有多大比例能在被引块原文里找到，低于阈值判不通过。
算重合度时有个坑要主动说出来：先剔掉在多数块里都出现的高频词元，否则「文件」「系统」这种词会让随便哪一块都及格。这跟 BM25 用逆文档频率压常见词是同一个道理。
校验不过怎么办：把具体原因写成反馈打回去重生成一次，只给一次机会；连着两版都编说明材料本来就不支持，该走拒答而不是第三次重试。另外校验必须拿原文比对，不能拿压缩或改写过的材料比对，否则「校验通过」保证不了用户点开看到的东西。
可预期的追问：为什么不让模型自己再检查一遍？因为自检和生成是同一个模型的同一种倾向，它对自己编的东西没有独立信息源；而编号核对是一个确定性判断，成本几乎为零、结果可复现，这两点自检都做不到。
Key points
Citations must be closed-set symbols such as block numbers, not free-text titles: verifiability comes from the closed set, not from wording.
Two gates: the number must exist, and the sentence must substantively overlap the cited block's original text, which is what catches real-number-wrong-content fabrication.
Strip terms that occur in most blocks before scoring overlap, or any citation will pass.
On failure, regenerate once with the concrete reason fed back; two bad drafts means refuse instead.
Always verify against the original text the user can open, never against a compressed or rewritten copy.
答题要点
引用必须是块编号这种闭集符号，不能是自由文本的文档标题——可验证性来自闭集，不来自措辞。
两道闸：编号存在性，以及这句话与被引块原文的实质重合度，后者才拦得住「编号是真的、内容对不上」。
算重合度前剔掉在多数块里都出现的高频词元，否则随便引哪一块都能及格。
校验不过就带着具体原因打回重生成一次，只给一次机会，两版都编就转拒答。
校验对象必须是用户能点开看到的原文，不是压缩或改写后的材料。
Does the ordering of retrieved passages in the context affect answer quality? If so, how would you order them?上下文里材料的排列顺序会影响回答质量吗？如果会，你会怎么排？
Common in ChinaCommon overseasBasic#context-assembly#prompt-engineering#ordering
How to reason about it · think before answering
This is a warm-up question, but 'sort by relevance descending' only earns half the credit. The interviewer wants to know whether you treat position itself as a variable.
State the conclusion first: it does matter. Models attend more reliably to material at the start and the end of the context, and are most likely to miss what sits in the middle. Plain descending order therefore parks your second-best passage in the worst spot.
Give the ordering: rank one first, rank two last, rank three second, rank four second-to-last, folding inward. Whatever ends up in the middle is by construction the least important, so the cost of it being skipped is smallest.
Round it out with the other assembly steps, which shows you have written this code: a deterministic tiebreaker (otherwise block numbers drift between runs and your logs stop matching), dedupe on normalised text, and a token budget that skips rather than stops when a block does not fit.
Expected follow-up: how would you verify this? Do not guess. Hold the question set fixed, vary only the ordering, and measure. Position effects differ by model and context length, so treat it as a parameter to measure on your own data rather than a universal law.
分析过程 · 先想清楚再作答
这是一道送分题，但答成「按相关性从高到低排」就只拿到一半分。面试官想听的是你知不知道位置本身是个变量。
结论先说：会影响。模型对上下文开头和结尾的材料明显更敏感，正中间的最容易被读漏。所以简单按分数从高到低顺排，等于把第二重要的材料放进了最不容易被读到的位置。
给出排法：第 1 名放开头、第 2 名放结尾、第 3 名放第二位、第 4 名放倒数第二位，依次往里收。这样按分数排下来越靠中间的块本来就越不重要，被读漏的代价最小。
顺带把排序之外的三道手续说全，显得你真的写过这段代码：同分要有决胜键（否则块编号会在两次运行之间飘，日志对不上）、要按归一化文本去重（同一段话常在手册和问答里各出现一次）、要有 token 预算并且塞不下时不要直接停。
可预期的追问：这个结论怎么验证？答案是别猜——固定一批问题，只改排列顺序跑对照，看指标差多少。位置效应在不同模型、不同上下文长度上强弱不一样，把它当成一个要在自己数据上量的参数，而不是一条普适定律。
Key points
Yes: material at the head and tail is used more reliably, the middle is most often skipped.
Put the strongest at both ends: rank one first, rank two last, rank three second, folding inward.
Assembly also needs a deterministic tiebreaker for stable numbering, dedupe on normalised text, and a token budget that skips oversized blocks instead of stopping.
The strength of the effect varies by model and context length, so measure it on your own data instead of quoting it as a law.
答题要点
会影响：开头和结尾的材料更容易被用上，正中间的最容易被读漏。
排法是最重要的放两端：第 1 名开头、第 2 名结尾、第 3 名第二位，依次往里收。
组装还要做三件事：同分给决胜键保证编号稳定、按归一化文本去重、控 token 预算且塞不下时跳过而不是终止。
位置效应的强弱因模型与上下文长度而异，要在自己的数据上做对照实验量出来，不能当普适定律照搬。
How do you set the refusal threshold for a knowledge-base assistant, and what does it cost you when the threshold is too high or too low?知识库问答的拒答阈值怎么定？定高了和定低了各自的代价是什么？
Common in ChinaCommon overseasDeep dive#refusal#thresholds#evaluation
How to reason about it · think before answering
What is really being tested: do you know that refusal is several rules rather than one threshold, and do you set thresholds from data. An answer that mentions only a score cutoff shows you have only touched the surface.
Break refusal into three rules with different timing. Score too low: decidable before generation, saving a model call. Sources conflict: also decidable before generation, by finding differing numbers about the same thing across blocks. You then either present both with their update dates, or pick the newer one when an authoritative signal backs it, such as meeting notes that flagged the discrepancy. Which of the two is a product decision, but silently letting the model pick is never an option. Question outside coverage: only decidable after generation, when citation verification leaves you with zero verified citations.
Stress that the three responses must read differently. 'Nothing relevant in the knowledge base, try rephrasing or check whether the document was ingested' is a different instruction to the user than 'we found related documents but none of them answers this'. Collapsing both into 'sorry, I don't know' throws away information.
Then the cost half. Too high: answerable questions get blocked, the user is told nothing was found while the material is in fact indexed. That is the most trust-damaging failure and it is nearly invisible in logs. Too low: weak passages enter the context and the model answers from irrelevant material, which is worse because the answer still looks cited.
How to set it: run a set of questions with known answers and known non-answers, look at where the two score distributions separate, and pick a point according to which error you fear more. Scores have no absolute scale, so the deliverable is the procedure, not the number.
Expected follow-up: what if one score threshold is not enough? Add signals rather than tuning the number: the gap between top and second score, the number of hits above threshold, and the post-generation verification result are all steadier than the raw score.
分析过程 · 先想清楚再作答
这题真正在考的是：你有没有意识到拒答不是一个阈值，而是好几条判据；以及你定阈值靠不靠数据。只谈一个分数阈值的回答，说明只做过最浅的一层。
先把拒答拆成三条线，它们的触发时机完全不同。检索分数太低：生成之前就能判，省一次模型调用。材料互相矛盾：也在生成之前判，代码在块之间找同一件事的不同数字，检出后要么并列两种说法与各自的更新日期，要么在有权威信号（比如一份点破了这条不一致的会议纪要）时按更新日期择一——选哪条是产品决策，但无论如何不能让模型自己悄悄挑一个。问题超出材料覆盖范围：只能在生成之后判，判据是跑完引用校验一条有效引用都没有。
强调三种话术必须不同。第一种要说「库里没有相关材料，换个说法或确认资料是否入库」，第三种要说「找到了相关文档但里面没有能直接回答的内容」——用户的下一步动作完全不同，混成一句「抱歉我不知道」等于把信息扔了。
再答代价这一半。定高了：能答的问题被挡在门外，用户看到查不到而材料其实在库里，这是最伤信任的一种错，而且它在日志里几乎不可见。定低了：低分噪声材料进上下文，模型拿着不相关的东西硬答，错误反而更隐蔽，因为回答看起来还带着引用。
怎么定：拿一批已知有答案和已知没答案的问题跑一遍，看两组的分数分布在哪里分开，按你更怕哪种错来取点。分数是没有绝对量纲的，换语料、换检索方式都要重定，所以真正要交付的是这套定阈值的流程，不是那个数字。
可预期的追问：单一分数阈值不够怎么办？答案是加判据而不是调数字——最高分与次高分的差、命中块数、以及生成后的引用校验结果，都是比原始分数更稳的信号。
Key points
Refusal is three rules, not one: low score and source conflict decided before generation, out-of-coverage decided after generation from the verification result.
On conflict, presenting both versions versus picking the newer one is a product decision; picking only holds up when an authoritative signal backs it.
The three responses must be worded differently because each implies a different next action for the user.
Too high blocks answerable questions; the user is told nothing exists while it does, which is the most damaging and least visible failure.
Too low lets weak passages in, producing errors that are harder to spot because the answer still carries citations.
Set it by comparing score distributions over answerable and unanswerable question sets, then choose based on which error is worse; re-tune whenever the corpus or retriever changes.
答题要点
拒答不是一条线而是三条：分数过低、材料冲突（都在生成前判）、超出材料覆盖范围（只能生成后按引用校验结果判）。
冲突检出后并列两说还是按更新日期择一，是产品决策；只有在有权威信号背书时择一才站得住，否则老实并列。
三种情况的话术必须不同，因为它们给用户的下一步动作不同。
定高了会把能答的问题挡住，用户看到查不到而材料其实在库里，最伤信任且日志里看不见。
定低了会让噪声材料进上下文，错误更隐蔽，因为回答看起来仍然带着引用。
定法是拿已知有答案与已知没答案的两组问题跑分数分布，按更怕哪种错取点；换语料或换检索方式都要重定。
In a streaming setup, how do you make sure nothing you have already sent needs to be retracted because its citation failed verification?流式输出的场景下，你怎么保证吐出去的内容不会因为引用校验失败而需要撤回？
Common in ChinaCommon overseasDeep dive#streaming#citation-verification#api-design
How to reason about it · think before answering
This tests a real architectural conflict: streaming wants the first token out early, citation verification cannot run until a statement is complete. Listen for whether the candidate names the trade-off and prices it.
Name the conflict: once a token reaches the browser you cannot take it back. Discovering at the end that the third sentence cited a fabricated block leaves you posting 'please ignore that last sentence', which is worse than not streaming at all.
Give the solution: buffer by sentence. As soon as a complete sentence lands, verify it, and only then emit it together with its verified citations; drop the whole sentence otherwise. The cost is that time-to-first-token becomes time-to-first-sentence, typically a few hundred milliseconds, which users barely notice, whereas a bad citation on screen costs trust.
Add two implementation details that prove you have built it. Streaming cannot use JSON output because JSON is only parseable once closed, so switch to plain text with inline markers, while keeping exactly the same verifier as the non-streaming path. Strip the markers out of the prose and send the numbers as structured data after verification.
Add the ordering point: the two rules decidable before generation, low score and source conflict, should be emitted before the stream starts, so the user never sees half an answer being withdrawn. The rule that needs generation shows up as 'no sentence was ever emitted', so close the stream with a refusal event.
Expected follow-up: does this kill the streaming feel? No. Sentence-level streaming is still visibly progressive on long answers. If you need finer granularity, stream a 'checking sources' placeholder, but never stream unverified prose.
分析过程 · 先想清楚再作答
这题在考一个真实的架构矛盾：流式要尽早出字，引用校验要等话说完才能核对。看回答里有没有出现「取舍」两个字，以及有没有把代价说清楚。
先说清矛盾在哪：一旦一个 token 发到了浏览器就撤不回来，你在末尾才发现第三句引用是编的，那句话已经在用户屏幕上了，只能补一句「刚才那句请忽略」，体验比不流式还糟。
给方案：按句缓冲。攒够一个完整句子就立刻校验一次，通过了才把这句连同已核实的引用发出去，没通过就整句丢掉。代价是首字延迟从一个 token 变成一句话，通常两三百毫秒，用户几乎察觉不到，而错误引用一旦上屏赔的是信任。
补两个实现细节，它们能证明你写过：流式模式没法用 JSON 输出（要等右花括号闭合才能解析），所以改成纯文本加行内标记，但校验必须和非流式共用同一套；标记要从正文里剥掉，正文保持干净，编号单独走校验再作为结构化数据发出去。
再补一条顺序上的讲究：生成前就能判的两条拒答线（分数过低、材料冲突）要在流开始之前发出去，用户不会先看到半句回答再被收回；生成后才能判的那条，在按句缓冲之下表现为一句都没发出来，收尾补一个拒答事件即可。
可预期的追问：那用户体验上的流式感是不是就没了？没有，句级流式在中文长回答里仍然是明显的渐进呈现；真要更细，可以在句子发出前先流一个「正在核对」的占位态，但不要流未校验的正文。
Key points
The conflict: emitted text cannot be recalled, while a citation can only be checked once its sentence is complete.
The fix is sentence-level buffering: verify each completed sentence, emit only if it passes, drop the whole sentence if it does not.
The cost is time-to-first-sentence instead of time-to-first-token, which is affordable and worth paying.
Streaming cannot use JSON, so use inline markers in plain text while sharing one verifier with the non-streaming path; strip markers from the prose and send numbers as structured data.
Emit pre-generation refusals before the stream opens; the post-generation one manifests as an empty stream and is closed with a refusal event.
答题要点
矛盾在于发出去的内容撤不回来，而引用只有一句说完才能核对。
解法是按句缓冲：攒够一句校验一次，通过才发，没通过整句丢掉。
代价是首字延迟从一个 token 变成一句话，这个代价必须付也付得起。
流式用不了 JSON，改纯文本加行内标记，但校验逻辑与非流式共用同一套；标记从正文剥出，编号作为结构化数据单独发。
生成前能判的拒答要在流开始之前发出去，生成后才能判的那条以「一句都没发」的形式收尾补事件。

D7 Week One Capstone: Assembling Six Days of Parts Into a One-Command Question-Answering Service, and a Retrospective

How would you draw the module boundaries of a RAG system, and which layer most needs to be swappable? Why?你会怎么划分一个检索增强生成系统的模块边界？其中哪一层最应该做成可替换的，为什么？
Common in ChinaCommon overseasBasic#architecture#modularity#embeddings
How to reason about it · think before answering
This question separates people who have maintained such a system from people who have only built a demo. Reciting the pipeline diagram is not an answer; where you cut it is.
Offer a reusable criterion first: cut where a layer is most likely to be replaced wholesale, not by lines of code or by tidy functional names.
Apply it. Embedding models change several times a year, and each change invalidates every stored vector, so that layer must be an interface. Storage may move from PostgreSQL to a dedicated vector database, and both ingestion and query talk through it, so it is the single shared boundary. Chunking changes daily during tuning, so it belongs in config, not in code.
Conclusion: the embedding layer is the one that must be swappable, because the swap is both likely and expensive, not because interfaces are good style.
Name the cost of abstraction too: every indirection is one more hop while debugging, so the test is whether the change will actually happen.
Expected follow-up: should the generation model be abstracted as well? Yes, but at lower priority, because swapping it does not force recomputation of stored data and rollback is cheap. It is a config value, not a layer.
分析过程 · 先想清楚再作答
这题考的是你有没有真的维护过这类系统。只按「解析、切块、检索、生成」复述一遍流程图，面试官会判定你只搭过 demo——流程图人人都会画，切口画在哪才是经验。
给一条可复用的判据再往下推：切口应该落在「将来最可能被整个换掉」的地方，而不是按代码量或者功能名称均分。
用它过一遍：embedding 一年会换好几次，换一次库里所有向量作废、必须全量重算，所以它必须是接口；存储可能从 PostgreSQL 换成专用向量库，而且摄取和查询都要通过它，所以它是两条链路的唯一交界；切块策略在调优期天天改，所以它必须是配置项而不是硬编码。
结论：最该做成可替换的是 embedding 那一层，理由不是「设计模式」，而是「换模型这件事真的会发生，且发生时代价极高」。
顺手点出抽象的代价：每多一层间接就多一次跳转和一份心智负担，所以判据是「那件事会不会真的发生」，不会发生的别抽象。
可预期的追问：那生成模型要不要也抽象？答案是要，但优先级低——换生成模型不需要重算任何存量数据，回滚也便宜，所以它是配置项而不是一层接口。
Key points
Lead with the criterion: cut where a layer is most likely to be replaced wholesale.
The embedding layer is the one to abstract: swapping models invalidates every stored vector and forces a full recompute.
Storage is the single boundary shared by ingestion and query, so define its interface before either implementation.
Chunking and retrieval routes belong in configuration because they change most often during tuning.
Abstraction costs indirection, so only abstract changes that will actually happen.
答题要点
先给判据：切口落在最可能被整体替换的那一层，不按代码量或功能名称均分。
embedding 是最该抽象的一层：换模型意味着存量向量全部作废、必须全量重算，代价高且真的会发生。
存储层是摄取与查询唯一的交界，接口要先定下来再谈两边实现。
切块与检索路数做成配置项，因为它们在调优期改动最频繁，改一次不该动代码。
抽象有成本，判据是那件事会不会真的发生；不会发生的抽象就是过度设计。
What should the ingestion path and the query path share, and what concretely goes wrong when you over-share?摄取链路和查询链路应该共享哪些代码？强行复用会带来什么具体问题？
Common in ChinaCommon overseasIntermediate#architecture#ingestion#retrieval
How to reason about it · think before answering
The word to notice is 'over-share'. The interviewer wants the boundary, not a recital of DRY.
Start from how the two paths differ. Ingestion is batch: tens of seconds, and a failure just means rerunning it. Query is online: hundreds of milliseconds, and a failure is visible to the user immediately. Error handling, timeouts and concurrency are simply not the same problem.
Hence the rule: share the interface, not the flow. The only genuinely shared thing is the storage interface, plus the embedding function signature.
Name the symptom of over-sharing: the extracted module fills up with isIngest branches, every change has to be verified on both paths, and eventually nobody dares touch it.
Add the one thing that truly must match: chunks and queries must be embedded by the same model. That is shared configuration, not shared code, and the model name belongs in the vector table so a silent mismatch is detectable.
Expected follow-up: what about chunking? The query path never chunks. Even when it needs a parent block, it reads it back through storage rather than importing the chunker.
分析过程 · 先想清楚再作答
题眼在「强行」两个字。面试官想看的是你能不能说出复用的边界，而不是背诵「不要重复自己」。
先说清两条链路的性质差异：摄取是批处理，几十秒跑完，失败重跑一遍就行；查询是在线请求，几百毫秒要出结果，失败用户当场看到。错误处理、超时、并发策略天然不同。
所以结论是：**共享接口，不共享流程**。两边唯一该共享的是存储层的那个接口，以及 embedding 的函数签名——注意后者共享的是签名和模型选择，不是调用流程。
给出强行复用的具体症状：抽出来的公共模块里开始出现 isIngest 这类分支，一个改动要同时验证两条链路，最后没人敢动它。
补一条真正必须一致的东西：给块算向量和给问题算向量必须用同一个模型。这不是复用代码，是复用配置——而且要把模型名写进向量表，否则模型换了没人发现，检索会静默地返回垃圾。
可预期的追问：那切块逻辑呢？查询侧压根不切块，所以它只属于摄取链路；真要在查询侧用到（比如 D11 的父子回填），走的也是存储层读回大块，不是把切块器搬过来。
Key points
Share the interface, not the flow: storage is the only boundary, plus the embedding signature.
The two paths have different error handling and latency budgets; batch can rerun, online must fail fast.
Over-sharing shows up as isIngest branches and changes that must be verified twice.
What must match is the model choice, not the code: record the model name alongside every stored vector.
Chunking belongs to ingestion only; the query path reads larger units back through storage.
答题要点
共享接口不共享流程：唯一的交界是存储层，加上 embedding 的函数签名。
两条链路的错误处理与延迟约束根本不同，批处理可以重跑，在线请求必须快速失败。
强行复用的症状是公共模块里长出 isIngest 分支，改一次要验两条链路。
必须一致的是模型选择而不是代码：块与查询要用同一个 embedding 模型，并把模型名记进向量表。
切块只属于摄取；查询侧需要大块时通过存储层读回，而不是把切块器搬过去。
What three checks would you run before shipping a retrieval QA service, and why those three?一个检索问答服务上线前你会做哪三项检查？为什么偏偏是这三项？
Common in ChinaCommon overseasIntermediate#production-readiness#citations#refusal
How to reason about it · think before answering
The discriminator is not how many checks you list but whether you can justify the three. Ten items with no ranking suggests you have never had to prioritise.
Derive them by consequence: the failures that are invisible to users and most damaging go first.
First, citations must be verifiable: every cited id resolves to a real chunk, and that chunk genuinely overlaps the sentence citing it. This ranks first because a wrong citation is undetectable by the user, and citations are the only source of trust this system has.
Second, refusal must actually fire: ask a question the corpus cannot answer and confirm the system says so instead of inventing. Also invisible, and one discovered fabrication zeroes out trust in the whole product.
Third, ingestion-to-retrieval consistency: freshly ingested documents are retrievable immediately, and the keyword and vector paths cover the same set. This guards against the 'one route finds it, the other does not' failure, which is the hardest to diagnose.
Expected follow-up: why not latency and cost? Because those failures are visible. Users complain about slowness and the bill reports overspending; nobody will ever report the three above.
分析过程 · 先想清楚再作答
这题的区分度不在你能列几项，而在你能不能说清「为什么是这三项」。列十项而每项都不给理由，反而说明你没有排过优先级。
推导方式是按后果排序：哪种故障用户看不出来、又损失最大，哪一项就该排在前面。
第一项是引用可查证：每条引用的编号都能回查到真实存在的块，且那一块确实与该句有实质重合。这一项排第一是因为引用错了用户根本发现不了，而它恰恰是这类系统唯一的信任来源。
第二项是该拒答时真的拒答：构造一个语料里没有答案的问题，看它是回那句拒答话术还是开始编。这一项也属于用户看不出来的故障，且一旦编造被发现，整个系统的可信度归零。
第三项是摄取到检索的一致性：摄取完之后新文档立刻能被检索到，且关键词与向量两路的覆盖数量对得上。这一项防的是「一路能查一路查不到」这种最难排查的故障。
可预期的追问：为什么延迟和成本不在前三？因为它们是**看得见**的故障——慢了用户会抱怨，贵了账单会告诉你；而上面三项不检查就永远不会有人告诉你。
Key points
State the ranking rule first: prioritise failures users cannot see but that cost the most.
Check one, verifiable citations: every id resolves to a real chunk that overlaps the sentence citing it.
Check two, refusal actually fires on a question the corpus cannot answer.
Check three, ingestion and retrieval agree: new documents are immediately retrievable on both routes.
Latency and cost matter but rank lower because those failures announce themselves.
答题要点
先给排序依据：优先检查用户发现不了、但后果最重的故障。
第一项引用可查证：编号能回查到真实的块，且该块与被引的那句话有实质重合。
第二项拒答生效：用一个语料里没有答案的问题验证系统会说查不到，而不是开始编。
第三项摄取与检索一致：新入库的文档立刻可检索，关键词与向量两路覆盖对得上。
延迟和成本重要但排在后面，因为它们是看得见的故障，会自己找上门。
What is the biggest risk in the RAG service you just assembled, and how would you prove that judgment?你刚拼出来的这个检索问答系统，现在最大的风险在哪里？你打算怎么证明这个判断？
Common in ChinaCommon overseasDeep dive#evaluation#risk-assessment#retrospective
How to reason about it · think before answering
There are two halves here and the second is the real question. Naming a risk is easy; giving a method that could falsify your own claim is what separates answers from opinions.
Rule out two common wrong answers: 'hallucination' is too vague to act on, and 'latency' mistakes a visible problem for the biggest one.
The biggest risk is the absence of evaluation. Chunk size, top-k, thresholds and route weights were all guessed, and that makes every other risk unverifiable: you cannot even say whether a change helped.
How to prove it: build a question set from the corpus with known answer documents, deliberately including unanswerable and multi-hop questions; implement recall and ranking metrics; produce a baseline for the current configuration; then move one parameter back and forth and watch whether the metrics move. If they do not move at all, the evaluation set is wrong, not the system.
Add the accounting rule: every optimisation reports three numbers, metric gain, latency added and cost added. A claim with only the first is not usable.
Expected follow-up: how large must the set be? Start with roughly twenty questions covering the main question types to catch obvious regressions, then grow toward the real distribution once you have actual user questions. Chasing size first only yields questions you invented yourself.
分析过程 · 先想清楚再作答
这题有两半，后半句才是题眼。说出一个风险不难，难的是给出一个能证伪你自己判断的方法——答不出后半句，前半句就只是意见。
先排除两个常见的错误答案：说「幻觉」太笼统，没有指向任何可动的地方；说「延迟」则是把看得见的问题当成最大风险。
真正的最大风险是**没有评估**：切块大小、取几条、门槛定多少、两路怎么加权，全是拍出来的。它最重要的地方在于它让所有其他风险都无法验收——你连「改了之后变好还是变坏」都说不出口。
怎么证明：先从语料反向出一份带标准答案文档的问题集，刻意掺进无答案问题和需要跨文档的多跳问题；再实现召回率与排序指标，给当前配置跑出一个基线；然后把一个参数来回改两次，看指标动不动。如果指标对参数完全不敏感，说明是评估集有问题，不是系统没问题。
补一句成本口径：每一项优化都要同时报三笔账——指标涨了多少、延迟涨了多少、钱涨了多少。只报第一笔的结论不能用。
可预期的追问：评估集多大才够？先做二十题能覆盖主要问题类型的小集，用它挡住明显的退步；等真实用户问题攒起来，再按真实分布扩到几百题。一上来就追求规模，只会得到一堆自己出的、跟真实用法无关的题。
Key points
The biggest risk is having no evaluation: every parameter was guessed, so no change can be judged.
Prove it by building a golden set with known answer documents, including unanswerable and multi-hop questions, then baseline the current configuration.
Validate the set itself by perturbing parameters: metrics that never move mean the questions are wrong.
Report three numbers per optimisation: metric gain, added latency, added cost.
Start small but well covered, then grow toward the real question distribution.
答题要点
最大的风险是没有评估：所有参数都是拍的，导致任何改动的好坏都无法判断。
证明方式是先建标准答案集，刻意包含无答案问题与多跳问题，再跑出当前配置的基线。
用参数扰动反过来验证评估集本身：指标对参数完全不敏感，说明题出得有问题。
每项优化同时报三笔账：指标、延迟、成本；只报指标的结论不能用。
评估集先小而全，覆盖问题类型即可，等真实问题攒起来再按真实分布扩大。

D8 Evaluation First: Building a Golden Set, Computing Recall and Ranking Metrics, Using a Model as Judge for Faithfulness

You need to build an evaluation set from scratch for a RAG system over a company knowledge base. How would you do it, and how many questions are enough?让你从零给一个公司知识库的 RAG 系统建评估集，你会怎么做？多少题才算够用？
Common in ChinaCommon overseasIntermediate#evaluation#golden-set#rag
How to reason about it · think before answering
The discriminator here is the direction you generate questions in, and whether you can justify a size rather than name one.
Go corpus-first: read each document and write the questions it can answer. The answer document is fixed at authoring time, so labeling is nearly free. Question-first gives you items whose answers nobody can locate.
Give the schema: question, answer document ids, and a type. At minimum three types - single-document, multi-hop, and unanswerable. Multi-hop counts as a hit only when every answer document makes it into the context; unanswerable items are scored on abstention, not recall.
Justify the size: 20 items separate 'broken' from 'usable' and are enough for a smoke gate; 100 to 200 are needed before a two-point delta means anything. Then grow the set - every production failure becomes a new item.
Mention cost and decay: roughly two hours for 20 items, and answer labels must be rechecked whenever the corpus changes, or the set rots and you misread the drop as a system regression.
Expected follow-up: how do you avoid overfitting to the eval set? Keep a held-out slice that never informs tuning, and refresh it from real production questions.
分析过程 · 先想清楚再作答
这题的区分度在「出题方向」和「规模的理由」两处。开口就说「找几百个用户真实问题」的，多半没真做过——真实问题的答案在哪篇文档里，没人标得出来。
先给方向：从语料反向出题，打开每一篇读它能回答什么，出题的那一刻答案文档就已经确定了，标注成本几乎为零。反方向（先想问题再找答案）会得到一堆自己都不知道答案的题。
再给结构：每题记问题、答案文档列表、类型三个字段；类型至少分单文档、多跳、无答案三类，并说明多跳必须全部答案文档命中才算命中，无答案不参与召回率而是考拒答。
规模的理由要给出来，不能只报一个数字：20 题能把「完全不能用」和「基本能用」分开，够做冒烟；100 到 200 题才有资格判断「涨了两个点」是真的还是噪声。上线之后每次线上出问题就把那个问题补进集合——评估集是长出来的。
补一句成本与保鲜：出题是人力活，20 题两小时是正常量级；语料更新后要复核答案文档还在不在，否则集合会悄悄腐烂，指标下跌你会误以为是系统坏了。
可预期的追问是「怎么防止评估集被过拟合」。答案是留一份不参与调优的保留集，并且定期从线上真实问题里补充新题，只用来验收不用来调参。
Key points
Author corpus-first so the answer document is known at authoring time.
Label every item with a type: single-document, multi-hop, unanswerable.
Multi-hop requires all answer documents; unanswerable items score abstention, not recall.
20 items for a smoke gate, 100 to 200 to trust small deltas, and keep growing it from production failures.
Hold out a slice that never informs tuning to avoid overfitting the set.
答题要点
从语料反向出题，出题时答案文档就已确定，标注成本最低。
每题标类型：单文档、多跳、无答案，三类缺一不可。
多跳要求全部答案文档命中；无答案不算召回率，考的是拒答。
20 题够冒烟，100 到 200 题才能判断小幅变化；线上故障持续补题。
留一份不参与调优的保留集，防止对评估集过拟合。
Recall, mean reciprocal rank, and normalized discounted cumulative gain - which failure mode does each one catch first, and what do you miss by watching only one?召回率、平均倒数排名、归一化折损累计增益，这三个检索指标分别在什么故障下会先掉下来？只盯一个会漏掉什么？
Common in ChinaCommon overseasIntermediate#retrieval-metrics#evaluation#ranking
How to reason about it · think before answering
This tests whether you know each metric's blind spot, not whether you can recite definitions. Layer them as 'did it show up / how high / how good overall' and you are halfway there.
Recall is boolean: is the answer document in the final context. It catches 'never retrieved', but it does not move when the answer slips from rank 1 to rank 8, as long as it still fits the budget.
MRR looks only at the rank of the first relevant hit, so ranking degradation shows up immediately. Its blind spot: one relevant item in the top ten scores exactly the same as five.
nDCG discounts every relevant hit in the top k by its position, so it tracks overall ranking quality and is the direct optimization target for reranking. Its blind spot is existence - it is zero both when nothing was retrieved and when ranking is terrible.
Conclusion: together they localize the failure. Recall drops means retrieval or chunking; recall flat but MRR down means ranking degraded, reach for a reranker; both stable but nDCG down means more noise crept into the top results.
Expected follow-up: what if a metric saturates? Make the questions harder - a saturated metric means the eval set lost its discriminative power, and further tuning is blind.
分析过程 · 先想清楚再作答
这题考的是「知不知道指标之间的盲区」，不是背定义。能把三者按「有没有 / 靠不靠前 / 整体好不好」分层的，基本就答对了一半。
推导链是这样的：召回率是布尔的——答案文档在不在最终上下文里。它对「压根没捞到」最敏感，但答案从第 1 名掉到第 8 名它一动不动，只要还在预算内。
倒数排名只看第一条相关结果的名次，所以「答案还在但被挤到后面」它立刻掉。反过来它有个盲区：前十条里有一条命中还是五条命中，它给的分完全一样。
归一化折损累计增益把前 k 名里每一条相关结果都按名次折算再累加，所以它对「整体排序质量」敏感，是重排最直接的优化目标。它的盲区是不告诉你「有没有」——召回率为零时它也是零，看不出是没捞到还是排得差。
结论：三个一起看才能定位故障层。召回率掉说明检索或切块出了问题，要动召回策略；召回率不动而倒数排名掉，说明排序退化，该上重排；两者都稳而 nDCG 掉，说明前几名里混进了更多噪声。
可预期的追问是「指标顶格了怎么办」。真实答案是把题目做难：指标撞天花板说明评估集失去区分度，这时候继续优化系统是在瞎调。
Key points
Recall answers 'did it make it into the context', sensitive to total misses, blind to rank shifts.
MRR answers 'how high is the first hit', sensitive to ranking degradation, blind to how many hits there are.
nDCG answers 'how good is the top k overall', the direct target for reranking, blind to existence.
Only the combination localizes the failure to retrieval, ranking, or noise.
State the hit criterion: context is packed against a token budget, not a fixed top-k.
答题要点
召回率管「有没有进上下文」，对完全没捞到最敏感，对名次变化不敏感。
平均倒数排名管「第一条排第几」，对排序退化最敏感，但分不清命中一条还是五条。
归一化折损累计增益管「前 k 名整体质量」，是重排的直接优化目标，但看不出有没有。
三者组合才能定位故障在召回层、排序层还是噪声层。
命中口径要说清：按 token 预算装上下文，不是按固定条数取前 k。
What systematic biases does an LLM judge have when scoring RAG faithfulness, and how do you detect them and prove your judge is trustworthy?用模型当裁判来评 RAG 的忠实度，有哪些系统性偏差？你怎么发现它们、又怎么证明你的裁判可信？
Common in ChinaCommon overseasDeep dive#llm-as-judge#evaluation#faithfulness
How to reason about it · think before answering
The second half of the question is the discriminator. Plenty of people can name position, length, and self-preference bias; few can say how they prove the judge is trustworthy.
Pair each bias with its mitigation: position bias - score pointwise instead of pairwise, and if you must compare, swap the order and call disagreement a tie; length bias - decompose into claims and score a ratio, so a longer answer grows its own denominator; self-preference - judge with a different vendor or tier than the generator.
Add two prompt-level requirements: fixed rubric anchors (spell out what 1.0, 0.6 and 0.3 mean, or the same input scores differently on different days) and forced structured output that quotes the unsupported sentences verbatim, which is what makes human review possible.
Proving trust has exactly one route: human spot-checks and an agreement rate. Stratify ten to thirty items across types, hits and misses, high and low judge scores; answer one binary question only - is anything here not in the material - and compare. Below 0.8 the judge's scores cannot gate a merge.
A detail that scores points: a very high agreement rate may mean your spot-check was too easy. If all ten sampled answers copy the material verbatim, agreeing is trivial and 100% says nothing about the judge.
Expected follow-up: can the judge itself break? Add probes - fixed inputs with known verdicts, one faithful and one obviously fabricated, checked on every run. An evaluation system fails silently: the numbers keep coming, they just stop meaning anything.
分析过程 · 先想清楚再作答
这题的题眼在后半句。能背出「位置偏好、长度偏好、自我偏好」三个名词的人很多，能说出「怎么证明可信」的很少——面试官要的是后者。
先把三个偏差和各自的缓解手段一一对应：位置偏好用逐条独立打分代替两两比较，非要比较就交换顺序跑两遍、结论不一致判平局；长度偏好用逐句判定加比例计分，写得越长分母越大，长度红利自动消失；自我偏好用跨供应商或跨档位的模型评判，生成和评判不同源。
再补两条提示词层面的：给死评分锚点，1.0 / 0.6 / 0.3 各自是什么必须写明，否则同一份输入不同天给的分都不一样；强制结构化输出并要求把没支撑的句子原样列出，这是人工复核的抓手。
证明可信只有一条路：人工抽检算一致率。分层抽十到三十条——各类型都要有、命中和没命中都要有、裁判给高分和低分都要有，只判一个二元问题（有没有材料外的内容），跟裁判的结论比对。低于 0.8 就不能拿它的分数做拦合并这类决策。
一个能加分的细节：一致率很高不一定是好消息。如果抽的十条都是「答案原样抄自材料」的简单题，判对是理所当然的，这时候 100% 说明的是抽检没难度，不是裁判可靠。
可预期的追问是「裁判本身会不会坏」。答案是给裁判写探针：喂几组已知正确答案的输入（照抄材料的、明显编造的），每次跑评估都验一遍——评估系统坏掉的方式最阴险，分数照常输出，只是不再有意义。
Key points
Three biases: position, verbosity, and self-preference, each with a matching mitigation.
Score pointwise rather than pairwise; decompose into claims and score a ratio to kill the length premium; never let the generator judge itself.
Pin rubric anchors in the prompt and force structured output that quotes unsupported sentences.
Establish trust through stratified human spot-checks and an agreement rate; below 0.8 the judge cannot gate merges.
Add probes with known verdicts so a broken judge is caught on every run.
答题要点
三个偏差：位置偏好、偏爱长答案、自己评自己，各自有对应的缓解手段。
逐条独立打分代替两两比较；逐句判定按比例计分抵消长度红利；生成与评判不同源。
提示词要给死评分锚点，并强制结构化输出、列出没支撑的句子。
可信度靠人工分层抽检算一致率，低于 0.8 不能用它做拦合并的决策。
给裁判本身写探针，每次跑评估都验一遍它有没有坏。
Why must a RAG evaluation set include questions the corpus cannot answer, and what does leaving them out hide?RAG 的评估集里为什么一定要放语料里没有答案的问题？不放会掩盖什么？
Common in ChinaCommon overseasBasic#evaluation#abstention#golden-set
How to reason about it · think before answering
It looks easy but really asks whether you have considered that the eval set itself can lie. 'To test the refusal path' is a pass; 'without them the worst failure is invisible in the report' is a full mark.
The derivation is one step: a system that always answers scores well on a set of answerable questions only. It stuffs context in, the model writes something, and the set has no column for 'should have refused'. The most dangerous failure simply does not appear.
Conclusion: unanswerable questions are the only thing that makes fabrication visible. They are excluded from recall and scored on abstention instead - did retrieval gate out every weak candidate, and did generation actually say the material does not cover this.
One authoring detail worth stating: unanswerable questions need strong distractor terms. Ask which browsers the web client supports when the corpus only says 'attach your browser and version when filing a ticket'. Without distractors retrieval returns nothing and you are testing your tokenizer, not your system.
Expected follow-up: what if the abstention rate is low? Check two layers - whether the retrieval score gate is effectively a no-op, and whether the generation prompt carries an explicit refusal instruction. You need both; a prompt alone is not a reliable gate.
分析过程 · 先想清楚再作答
这题看着简单，实际是在问「你有没有想过评估集本身也会说谎」。答成「为了测试拒答功能」只算及格，答出「不放会让某个故障在报表上完全不可见」才是满分。
推导只有一步：一个只会硬答的系统，在只有可答问题的评估集上能拿到很高的分——它每次都塞材料给模型，模型每次都编一段话，而评估集根本没有「应该拒答」这一栏。于是最危险的故障在报表上是不存在的。
结论：无答案问题是唯一能让「乱编」显形的东西。它不参与召回率，它的指标是拒答率——检索侧有没有把不够格的候选全挡下来，生成侧有没有真的说出「资料里没有」。
出题上有个必须说的细节：无答案问题必须留强干扰词，比如问「网页端支持哪些浏览器」而语料里恰好有一句「提交工单请附上浏览器与版本」。没有干扰词的无答案题检索器一条都捞不到，你测出来的是分词器不是系统。
可预期的追问是「拒答率低怎么办」。分两层查：先看检索侧的门槛是不是形同虚设（分数阈值定得太低，不相干的块也过关），再看生成侧的提示词有没有明确的拒答指令，两层都要有，只靠提示词兜是不牢的。
Key points
An all-answerable eval set makes 'answers confidently when it should not' completely invisible.
Unanswerable items are scored on abstention, not recall, and you check both the retrieval gate and the generation refusal.
Author them with strong distractor terms, or retrieval returns nothing and you are testing the tokenizer.
Keep them at roughly 15% or more of the set, alongside multi-hop items, as the coverage floor.
A low abstention rate splits into two causes: a no-op retrieval score gate, or a missing refusal instruction in the prompt.
答题要点
只有可答问题的评估集，会让「不知道也硬答」这个故障完全不可见。
无答案问题不算召回率，它的指标是拒答率，检索侧和生成侧各看一层。
出题必须留强干扰词，否则检索器一条都捞不到，测的是分词器。
建议无答案题占比不低于评估集的一成五，跟多跳题一起构成覆盖度底线。
拒答率低要分两层查：检索门槛是否形同虚设，生成提示词有没有拒答指令。

D9 Hybrid Search and Reranking: Two-Path Retrieval, Reciprocal Rank Fusion, Then Re-Ranking the Top Results With a Cross-Encoder

Why do hybrid retrieval systems usually use reciprocal rank fusion instead of normalizing both scores and adding them with weights? When does the weighted approach break down?混合检索为什么普遍用倒数排名融合，而不是把两路分数归一化之后加权相加？加权那条路在什么情况下会失控？
Common in ChinaCommon overseasIntermediate#hybrid-search#rank-fusion
How to reason about it · think before answering
The hinge word is `scores`. Answering `RRF is simpler` is reciting a concept; the interviewer wants to hear that you know why the two scores are not comparable in the first place.
Start with scale: BM25 is an unbounded sum of log terms, and on one index the top hit can range from 5 to 50 depending on the query; cosine is pinned between -1 and 1. Adding those two readings is meaningless.
Then name the silent failure of normalization: dividing by the per-route maximum makes the denominator float with the query. For a question with no answer in the corpus, the vector route's best hit may score 0.09 and still normalize to a perfect 1.0, entering the fusion at full weight. You think you are comparing relevance; you are comparing `tallest among the short`.
Then the maintenance cost of weights: a 1-to-0.6 ratio has to be tuned against an eval set, tuning two routes is a 2-D search, adding multi-query retrieval makes it 4-D or 5-D, and swapping the embedding model invalidates all of it. RRF has a single k, and the default of 60 rarely needs touching.
Conclusion: rank is the only thing the two routes share. RRF throws the scores away on purpose so that an incomparable quantity cannot mislead it.
Expected follow-up: what does k do? It flattens — the larger k is, the smaller the gap between the top few ranks, so `ranked well by both routes` outweighs `ranked first by one route`, which is exactly the cross-validation effect hybrid retrieval is after. A second follow-up on ties: you must fall back to sorting by document id, or ranks drift between runs and every eval number wobbles with them.
分析过程 · 先想清楚再作答
这题的题眼在「分数」两个字。只答「RRF 更简单」是背概念，面试官想听的是你知道分数为什么不可比。
先给量纲差异：BM25 是一堆对数项累加，没有上界，同一套索引里不同查询的第一名可以从 5 分到 50 分；余弦被钉死在负一到正一。两个读数相加没有意义。
再点出归一化的静默失败：除以本路最高分之后，分母随查询浮动。一个语料里根本没有答案的问题，向量那一路最高分只有 0.09，归一化之后照样是满分 1.0 带权重进融合——你以为在比相关性，其实在比「本路矮子里有多高」。
然后是权重的维护成本：1 比 0.6 这个配比要靠跑评估调出来，两路是二维搜索，加上多路查询就是四维五维，而且换一个 embedding 模型全部作废。RRF 只有一个 k，而且 60 这个默认值几乎不用动。
结论：名次是两路唯一可比的东西。RRF 主动扔掉分数，是为了不被不可比的量误导。
可预期的追问：那 k 是干什么的？答 k 是压平器——k 越大，头几名之间的差距越小，于是「两路都排进前列」比「一路排第一」更有分量，这正是混合检索想要的交叉验证效果。再追问同分怎么办，答必须按文档 id 兜底排序，否则跨次运行名次会飘、评估数字跟着抖。
Key points
BM25 is unbounded, cosine is bounded; the two scales are not comparable, so adding them is meaningless.
Per-route max normalization has a denominator that floats with the query, so the least relevant hit of an unanswerable query still normalizes to 1.0.
Weights must be tuned against an eval set, the search is high-dimensional once you add routes, and swapping models invalidates it; RRF has a single constant k.
RRF consumes only the ordered id list from each route, because rank is the one thing the routes share.
Larger k rewards `ranked well by both routes`; ties must fall back to document id so results are reproducible.
答题要点
BM25 无上界、余弦有界，两个量纲不可比，直接相加没有意义。
按本路最高分归一化的分母随查询浮动，无答案的查询里最不相干的结果也能拿到满分。
权重要跑评估调，路数一多就是高维搜索，换模型还得重来；RRF 只有一个常数 k。
RRF 只吃每一路的有序 id 列表，名次是两路唯一可比的东西。
k 越大越奖励「两路都排进前列」；同分必须按 id 兜底排序才可复现。
Why is a cross-encoder more accurate than a bi-encoder? And if it is more accurate, why not just use it to search the whole corpus directly?交叉编码器为什么比双编码器准？既然更准，为什么不干脆拿它直接检索全库？
Common in ChinaCommon overseasBasic#cross-encoder#bi-encoder
How to reason about it · think before answering
This is a giveaway question, but the discriminating half is the second part. Saying `cross-encoders are slow` is not enough; you have to point at the structural reason.
Start with the structure: a bi-encoder encodes query and document **separately** into vectors that never meet until a single dot product at the end; a cross-encoder concatenates query and document into one sequence, so every attention layer lets query tokens attend to document tokens.
That yields the accuracy gap: a bi-encoder must compress a document into one fixed-length vector, and compression loses information — the binding between `Zhou Min` and `platform team lead` may not survive. A cross-encoder does not compress; it aligns them on the spot.
The answer to the second half hides in the same structure: bi-encoder document vectors can be computed **offline** and indexed, so query time is just a vector search. A cross-encoder has nothing to precompute — N documents means N forward passes. Reranking a 100k-chunk corpus means pushing the entire corpus through a model on every question.
So the engineering split is a division of labor: recall pulls a small batch out of the whole corpus (cheap, indexable), reranking fixes the order of that batch (expensive, accurate). The default is to rerank only the top 20 after fusion.
Expected follow-up: is there a middle path? Yes — late interaction, where token-level document representations are precomputed and the interaction happens at query time. Accuracy and cost land between the two, at the price of a much larger index.
分析过程 · 先想清楚再作答
这是一道送分题，但送分题的区分度在第二问。只答「交叉编码器慢」是不够的，要说清慢在结构上的哪一处。
先给结构差异：双编码器把查询和文档**各自**编码成向量，两者从头到尾没有见过面，最后只靠一次内积凑到一起；交叉编码器把查询和文档拼成一段文本一起过模型，每一层注意力都能让查询的词去看文档的词。
由此推出准确率差异的来源：双编码器要把一篇文档压成一个固定长度的向量，压缩必然丢信息，「周敏是平台组组长」里两个词的绑定关系未必留得下来；交叉编码器不压缩，它当场对齐。
第二问的答案就藏在同一个结构里：双编码器的文档向量**可以离线算好**，查询时只做向量检索；交叉编码器没有任何东西能预先算好，N 篇文档就要跑 N 次前向。十万块的语料重排一遍，等于每次提问都把整个库过一遍模型。
所以工程上的定位是分工：召回负责在全库里捞出一小批（便宜、可索引），重排负责把这一小批的顺序改对（贵、准）。默认只重排融合后的前 20 条。
可预期的追问：有没有中间路线？答有——后期交互（late interaction）那一类，文档侧提前算好词级表示、查询侧当场做交互，精度和成本都在两者之间，代价是索引体积大得多。
Key points
A bi-encoder encodes both sides separately and joins them with one dot product; a cross-encoder concatenates them so attention can align across the pair.
The accuracy gap comes from compression: a bi-encoder squeezes a whole document into one vector and loses bindings; a cross-encoder does not compress.
Bi-encoder document vectors can be computed offline and indexed; a cross-encoder has nothing to precompute.
Reranking the full corpus means running every chunk through a model on every question, so cost scales linearly with corpus size.
The standard split is recall plus rerank, with reranking applied only to the top few dozen after fusion.
答题要点
双编码器各自编码、最后一次内积；交叉编码器把查询和文档拼在一起过模型，注意力可以跨两者对齐。
准确率差异来自压缩：双编码器把整篇文档压成一个向量，绑定关系会丢；交叉编码器不压缩。
双编码器的文档向量能离线算好并建索引，交叉编码器没有任何东西可以预先算好。
全库重排等于每次提问把整个语料过一遍模型，成本随语料规模线性增长。
标准分工是召回加重排，重排只作用于融合后的前几十条。
You replaced pure vector retrieval with hybrid search plus reranking, and after shipping it your eval metrics went down. How do you investigate?你把纯向量检索换成了混合检索加重排，上线之后评估指标反而掉了。你会怎么排查？
Common in ChinaCommon overseasDeep dive#hybrid-search#evaluation
How to reason about it · think before answering
This question tests whether you have actually done stage-by-stage attribution. Answering `I would tune the weights and see` loses — that is guessing, not investigating.
Step one is to run the stages apart, not to change code: pure keyword, pure vector, hybrid, and hybrid plus rerank, all on the **same eval set with the same context budget**. Whichever stage the drop appears in is where you look, and this alone separates `fusion is broken` from `reranking is broken`.
Step two asks a specific question: did recall drop, or did the ranking metrics drop? A recall drop means the answer never entered the context at all — a candidate-pool or budget problem. Ranking metrics dropping while recall holds means the answer is still there but pushed down — a fusion-weight or rerank-model problem. The two failures have completely different fixes.
A third common root cause is recall depth. This knob runs against intuition: going deeper is not safer, it lets noise vote too. On a 134-chunk corpus I measured that narrowing each route from 50 to 5 took hybrid recall from 87.5% back to 93.8% and multi-hop from 50% to 75%, while nDCG fell by almost 0.1. The metrics fight each other, so decide which one the product needs first.
A fourth root cause is that the eval protocol quietly changed. Touch the context budget, the hit rule, or the candidate depth, and the old and new numbers stop being comparable — in which case the `drop` may not be a drop at all.
Expected follow-up: how do you avoid this next time? Make the four-way comparison a single command, store the previous report as a baseline, and fail the build with a non-zero exit code on regression. That is precisely why evaluation comes before optimization.
分析过程 · 先想清楚再作答
这题考的是你有没有真的做过分阶段归因。答「调一下权重再看看」就输了——那是在猜，不是在查。
第一步是拆档跑，不是改代码：纯关键词、纯向量、混合、混合加重排四档在**同一份评估集、同一个上下文预算**下各跑一遍。指标掉在哪一档就在哪一档找原因，这一步能立刻区分「融合坏了」和「重排坏了」。
第二步问一个具体问题：掉的是召回率还是排序指标？召回率掉说明答案根本没进上下文，是候选池或者预算的问题；排序指标掉而召回率没动，说明答案还在、只是被挤到了后面，那是融合权重或重排模型的问题。这两类故障的解法完全不同。
第三个常见根因是召回深度。每路取多少条这个旋钮方向反直觉：取深了不是更保险，是把噪声也一起投了票。我在一份 134 块的语料上实测过，每路从取 50 收到取 5，混合那一档的召回率从 87.5% 回到 93.8%、多跳档从 50% 回到 75%，而 nDCG 反而掉了近 0.1——两个指标会打架，先想清楚业务要哪个。
第四个根因是评估口径被悄悄改了。上下文预算、命中判定、候选池深度只要动过一个，新旧数字就不可比，这时候「掉了」可能根本不是真的掉了。
可预期的追问：怎么防止下次再踩？答把四档对照做成一条命令、把上一版报告存成基线、指标退步就以非 0 退出码拦住合并——这就是评估要先于优化的原因。
Key points
Run all four configurations separately for attribution, on one eval set with one context budget, before touching any parameter.
Separate a recall drop from a ranking drop: the first is a candidate-pool or budget issue, the second is a fusion or rerank issue.
Check recall depth: taking too many per route lets noise vote, and narrowing it can bring recall back.
Confirm the eval protocol did not change; touching budget, hit rule, or candidate depth makes old and new numbers incomparable.
Freeze the four-way comparison into one command plus a baseline report, and block merges on regression.
答题要点
先拆档跑四种配置，在同一份评估集和同一个上下文预算下归因，不要一上来就调参。
区分召回率掉与排序指标掉：前者是候选池或预算问题，后者是融合或重排问题。
查召回深度：每路取太深会把噪声也投进融合，收窄反而可能救回召回率。
确认评估口径没被改：预算、命中判定、候选池深度动过一个，新旧数字就不可比。
把四档对照固化成一条命令加一份基线报告，指标退步直接拦住合并。
Adding a reranker costs you 200 ms of extra latency per question plus a per-search fee. How do you decide whether that spend is worth it?加上重排之后每次提问多了两百毫秒延迟，还多了一笔按次计费的开销。你怎么判断这笔钱该不该付？
Common in ChinaCommon overseasDeep dive#rerank#cost-tradeoff
How to reason about it · think before answering
This question tests whether you can translate a technical choice into a business judgment. Answering `check whether the metrics went up` covers only a third of it.
Split it into three ledgers: how much the metrics moved, how much latency grew, and how much money it costs. All three must be reported together; a proposal with only the first will not survive review.
For the first ledger, be specific about **which** metric reranking improves. Reranking changes the order, not the candidate set — it cannot fix `the answer was never retrieved`. If your recall is the bottleneck, add a retrieval route or adjust recall depth first; the 200 ms buys nothing.
For the second, ask where those 200 ms land. They sit synchronously between retrieval and generation, with the user waiting; but if a streaming generation follows and time-to-first-token is already a second or two, the relative cost is small. In an as-you-type search box, 200 ms is fatal.
For the third, note the billing unit: rerankers usually charge per search rather than per token, so sending a few more candidates barely changes the bill — what is expensive is the number of questions. That points optimization at reducing query volume (caching, intent routing) rather than at trimming the candidate list.
Expected follow-up: what if you simply cannot afford it? Three paths — rerank only queries classified as hard (intent routing), cache results, or self-host an open-weights cross-encoder to convert per-call fees into fixed compute cost.
分析过程 · 先想清楚再作答
这题考的是你会不会把技术选择翻译成业务判断。只答「看指标涨没涨」只答了三分之一。
先把账拆成三笔：指标涨了多少、延迟涨了多少、钱涨了多少。三笔必须一起报，只报第一笔的方案在评审会上过不去。
第一笔要问清楚重排改善的是**哪个**指标。重排改的是顺序，不是候选集合——它救不了「答案压根没被召回」这种故障。如果你的召回率本来就不够，先去加召回路数或者调召回深度，重排这两百毫秒是白花的。
第二笔要看这两百毫秒落在哪。它是同步卡在检索之后、生成之前的，用户全程在等；但如果后面接的是一个流式生成、首字节本来就要一两秒，这两百毫秒的相对占比就小得多。反过来，如果这是一个自动补全式的即时搜索框，两百毫秒就是致命的。
第三笔要注意计价单位：重排普遍按检索次数计价而不是按 token，所以「多送几条给它排」几乎不涨钱，真正贵的是提问次数本身。这直接决定了优化方向是压提问量（缓存、意图路由）而不是压候选数。
可预期的追问：如果就是付不起怎么办？答三条路——只对判定为复杂的查询走重排（意图路由）、把结果缓存起来、或者换成自部署的开源交叉编码器把按次付费变成固定的算力成本。
Key points
Report all three ledgers together: metric gain, latency growth, cost growth; a proposal missing one is incomplete.
Confirm whether the bottleneck is ordering or recall first; reranking only reorders and cannot rescue an answer that was never retrieved.
Judge the latency by where it lands: it is small relative to a streaming generation, but fatal in an as-you-type search box.
Rerankers bill per search rather than per token, so cost scales with question volume, not candidate count.
If it is unaffordable: route only hard queries to the reranker, cache results, or self-host an open-weights cross-encoder.
答题要点
三笔账一起报：指标增量、延迟增量、成本增量，缺一笔方案就不完整。
先确认瓶颈是排序还是召回：重排只改顺序，救不了没被召回的答案。
延迟要看落在哪：流式生成场景下相对占比小，即时搜索框里两百毫秒就是致命的。
重排按检索次数计价而不是按 token，涨钱的是提问量而不是候选条数。
付不起时的三条路：意图路由只对难查询重排、结果缓存、换自部署的开源交叉编码器。

D10 Query-Side Optimization: Rewriting, Hypothetical Document Embeddings, Multi-Query, Step-Back Prompting, and Intent Routing

Why does HyDE (hypothetical document embeddings) work, and when does it steer retrieval in the wrong direction?假设文档嵌入（HyDE）为什么有效？它在什么情况下会把检索带偏？
Common in ChinaCommon overseasIntermediate#hyde#query-transformation#retrieval-quality
How to reason about it · think before answering
The tell is in the second half. Anyone can recite why HyDE works; only someone who has run it on real data can say when it hurts.
Give the mechanism first: dense retrieval compares semantic similarity, but a user's question and a policy paragraph differ in register, syntax and vocabulary. HyDE has the model draft a fake passage that looks like the target document, then retrieves with that vector — effectively moving the query into the documents' register.
Then kill the common misreading: the factual accuracy of the draft does not matter, because it is never shown to the user. It only contributes a direction in embedding space.
Two failure modes. The model invents an over-specific field or process name that does not exist in the corpus, and the vector chases something imaginary. Or the corpus genuinely has no answer, and the fabricated passage finds plausible-looking neighbours anyway — abstention rate drops and hallucination rate climbs.
Pair the risk with a mitigation: gate admission on each retriever's raw score, never on the fused score (fused scores are relative, so even the worst batch tops out at 1.0); and treat the hypothetical document as a second query fused with the original rather than a replacement, so a bad draft can only dilute the signal, not erase it.
Expect the follow-up on cost. The draft runs to a hundred-plus output tokens, an order of magnitude more than a rewrite, and it doubles retrieval calls. That is why it belongs in an A/B queue, not in the default config.
分析过程 · 先想清楚再作答
这题的题眼在后半句。前半句网上到处都能抄到，能不能说清「什么时候不该用」才是区分度所在——只答前半句的人，多半没在真实语料上跑过。
先给机制：向量检索比的是语义相似度，而用户的疑问句和文档里的制度条文在文体、句式、用词上都不同类。HyDE 先让模型编一段「长得像目标文档」的假文本，用它的向量去找邻居，等于把查询搬进了文档所在的那个语域。
紧接着点破一个常见误解：这段假文本的**事实对不对根本不重要**，因为它不给用户看，只贡献一个向量方向。理解到这一层，才算真懂它为什么不怕模型瞎编。
带偏有两种典型情况。一是模型编得太具体，给出语料里根本不存在的字段名或流程名，向量朝着一个不存在的方向去了；二是语料里压根没有答案，本该拒答的问题被编出来的假文档匹配到几个「看起来挺像」的邻居，拒答率掉下去、瞎编率涨上来。
说完风险要给对策，这一步最见工程经验：门槛卡在**每一路检索器的原始分**上而不是融合分上（融合分是相对的，最不相干的一批也能拿最高分）；以及把假设文档当成**第二个检索式与原问题融合**，而不是直接替换原问题——替换在模型编歪时会把原问题的信号一起丢掉。
可预期的追问是「它多花多少钱」。答：假设文档要写上百字，输出 token 是查询改写的十几倍，是查询侧四种手法里最贵的一次调用，而且检索次数翻倍。所以它通常不该默认打开，应该进 A/B 队列。
Key points
It works by register alignment: a question and a policy paragraph sit in different neighbourhoods, and the fake passage moves the query into the document's.
The draft's factual accuracy is irrelevant — it only supplies a direction and is never shown to the user.
It misfires when the model invents over-specific details, or when the corpus has no answer and the fabrication finds plausible neighbours anyway.
Two guardrails: gate on raw per-route scores, not fused ones; fuse the hypothetical document with the original query instead of replacing it.
It is the most expensive query-side technique (long output plus doubled retrievals), so keep it off by default and A/B it.
答题要点
有效的原因是语域对齐：疑问句和制度条文本来不在一个语义邻域，假设文档把查询搬到了文档那一侧。
假文本的事实对错不重要，它只贡献一个向量方向，不展示给用户。
带偏的两种情况：编得太具体，追一个语料里不存在的方向；本该拒答的问题被假文档匹配上，拒答率下降。
两条护栏：门槛卡原始分不卡融合分；把假设文档当第二个检索式融合，而不是替换原问题。
成本上它是查询侧最贵的一项（长输出加检索次数翻倍），默认关闭、按场景 A/B。
How do you handle coreference in multi-turn RAG, and what is the classic failure when you skip it?多轮对话里怎么处理指代？不做指代消解最典型的翻车场景是什么？
Common in ChinaCommon overseasBasic#coreference#multi-turn#query-rewriting
How to reason about it · think before answering
This is a warm-up question, but there is still a gap between answers. Saying "just concatenate the history into the query" invites a follow-up about growing histories that most candidates cannot handle.
State the mechanism: insert a short rewrite call before retrieval that takes the last few turns plus the current question and returns one retrieval-ready line. Set temperature to 0 so the same input always yields the same query, and forbid the model from answering the question in the prompt.
Explain why concatenation is worse: history grows without bound, filler words dilute inverse document frequency, and the previous answer leaks in — you end up retrieving an answer with an answer. The rewriter emits one sentence, not a transcript.
Make the failure concrete. Turn one: "who must sign off on this operation?" Answer: "the platform team lead." Turn two: "what is that person's name?" Retrieved unresolved, not a single candidate clears the admission gate and the system refuses — even though the corpus contains the answer. The failure is not a wrong answer, it is a false "not found" right after the user's own question.
Add the ordering trap: rewrite before intent routing. A pronoun is a classic multi-hop signal, so an unresolved query gets routed to the expensive path for nothing; after rewriting it is an ordinary single-hop question. Multi-query and step-back must also sit downstream of the rewrite, or one unresolved pronoun becomes three.
Expect "how do you decide when to rewrite?" Trigger on short queries, pronouns and elliptical follow-ups; skip on a clearly new topic. The check is nearly free and removes most of the calls.
分析过程 · 先想清楚再作答
这是一道送分题，但送分题也有高下之分：只说「把历史拼进查询里」的答案，会被追问一句「历史越拼越长怎么办」就卡住。
先把做法说清：在检索之前加一次很短的改写调用，输入是最近几轮对话加本轮问题，输出是一行可以直接检索的检索式；温度设 0 保证同一句话每次改成同一个结果，并在提示词里明确禁止模型顺手回答问题。
为什么不是「把历史整个拼进查询」：历史越拼越长，噪声词把逆文档频率摊薄，检索反而更差；而且历史里包含上一轮的答案，等于拿答案去检索答案。改写的产出是一句话，不是一段历史。
最典型的翻车场景要举实例：上一轮问「这个操作必须由谁审批」，答「必须由某某组组长审批」；这一轮问「这个人叫什么名字」。不消解直接检索这七个字，实测是**一条候选都过不了门槛，系统只能拒答**。注意失败方式不是答错，是「明明语料里有答案却说找不到」，用户体验是崩塌式的。
补一条顺序上的坑：改写必须在意图路由**之前**。「这个人」是典型的多跳信号词，路由看到它会判成多跳、白跑一轮；改写之后它只是个普通单跳问题。同理，多路查询、后退提问也都要建立在改写后的那句话上，否则错误被放大好几倍。
可预期的追问是「怎么知道要不要改写」。答：短问题、含指代词、含省略（「那审计日志呢」）时才触发，纯新话题跳过——这一步很便宜，但能省掉一大半调用。
Key points
Add a short rewrite call before retrieval: last few turns plus current question in, one retrieval line out, temperature 0, answering explicitly forbidden.
Do not splice the whole history into the query — it grows unbounded, dilutes IDF, and leaks the previous answer into the search.
Classic failure: an unresolved pronoun means no candidate clears the gate, so the system refuses a question the corpus can answer.
That false "not found" hurts more than a wrong answer, since the user just asked about the same thing.
Order matters: rewrite first, then route; multi-query and step-back both build on the rewritten query.
答题要点
在检索前加一次短改写调用，输入最近几轮加本轮问题，输出一行检索式，温度 0，禁止模型回答问题。
不要把历史整段拼进查询：越拼越长、噪声稀释逆文档频率，还会拿上一轮的答案去检索。
典型翻车：上一轮的「这个人 / 他 / 那个」不消解，检索一条都过不了门槛，系统在有答案的情况下拒答。
失败方式是「假的查不到」，比答错更伤体验，因为用户刚刚才问过同一件事。
顺序：先改写、再路由，多路查询与后退提问都建立在改写后的查询上。
Query rewriting adds a model call per question and doubles end-to-end latency. How do you decide whether it is worth paying?上线查询改写之后每问多了一次模型调用，端到端延迟涨了一倍，你怎么判断这笔开销值不值？
Common in ChinaCommon overseasDeep dive#cost-tradeoff#latency#query-rewriting
How to reason about it · think before answering
This question is about turning an engineering judgement into numbers. "Rewriting obviously helps quality" is a fail — the candidate never measured the gain.
Start with one question that nearly settles it: which slice of traffic does the gain land on? Query rewriting buys almost nothing on single-turn questions (we measured identical metrics with it on and off across 20 single-turn items); the entire payoff is in follow-up turns. So step one is to pull the share of multi-turn sessions from production logs.
Step two is to lay out all three ledgers, because one alone cannot support a decision: how much the metrics moved on a fixed golden set, how much latency grew (a rewrite is a short-output task, so a cheap fast model often costs a few hundred milliseconds rather than doubling anything), and how many extra calls were added — one model call for rewriting versus one call plus several retrievals for multi-query is a completely different cost shape.
Step three is to price the cheaper variants before deciding: rewrite only when a trigger fires (short query, pronoun, ellipsis), cache rewrites per session, and run a small model instead of the main one. These usually remove most of the cost while keeping the gain.
Land on a usable rule: gain times affected traffic share, divided by added latency and cost, ranked against your other candidate optimisations. Rewriting usually ranks high because its failure mode is a false "not found" immediately after the user's own question — an abandonment-grade experience bug, not a few metric points.
Expect "what if the latency genuinely is unacceptable?" Fire the rewrite and the first retrieval in parallel: search with the raw query immediately, search again when the rewrite returns, and fuse both rankings. You pay a max instead of a sum, at the cost of one extra retrieval.
分析过程 · 先想清楚再作答
这题考的是「能不能把工程判断落到数字上」。凡是回答「改写当然要做，能提升效果」的，一律判为没做过——他连收益是多少都没量。
先问自己一句：**收益出现在哪一类流量上**。这一条几乎决定了答案。查询改写在单轮问答上的收益接近零（我们在 20 道单轮题上实测开关它指标一模一样），收益全在多轮追问。所以第一步是去线上日志里查多轮会话占比，占比很低的话这笔钱不该花在全量流量上。
第二步是把三笔账摆齐，缺一笔就不能下判断：指标涨了多少（用固定的标准答案集跑，不要用感觉）、延迟涨了多少（改写是短输出任务，可以换便宜快的那一档模型，往往只多两三百毫秒而不是翻倍）、多了几次调用（改写是一次，多路查询是一次调用加几次检索，成本结构完全不同，别混着算）。
第三步是找**便宜的替代路径**再比一次：只在命中触发条件时才改写（短问题、含指代词、含省略），纯新话题直接跳过；改写结果按会话缓存；用小模型跑改写而不是主模型。这三招通常能把这笔开销压掉一大半，而收益几乎不掉。
结论要落成一条可执行的判据：**收益乘以受影响流量占比，除以增加的延迟与成本**，跟你手上其他候选优化排个序。改写通常能排到很前面，因为它的失败方式是「用户明明追问同一件事却被告知查不到」，那是会直接导致弃用的体验故障，不只是指标掉几个点。
可预期的追问是「延迟真的不能接受怎么办」。答：把改写和第一次检索**并行发**，用原查询先检索一路，改写回来后再补一路，两路用倒数排名融合合起来——延迟只多一个 max 而不是一个加法，代价是多一次检索。
Key points
Locate the gain first: rewriting is near-zero on single-turn traffic and pays off on follow-ups, so start from the share of multi-turn sessions.
All three ledgers are mandatory: metric delta on a golden set, added latency, added calls and token cost.
Try the cheap variants before deciding: conditional triggering, per-session caching, and a small model for the rewrite.
Decide on gain times affected traffic share over added latency and cost, then rank it against your other optimisations.
If latency is a hard constraint, fire the rewrite in parallel with the first retrieval and fuse both rankings, turning a sum into a max.
答题要点
先定位收益落在哪一类流量：改写在单轮上接近零收益，价值全在多轮追问，先查多轮会话占比。
三笔账缺一不可：标准答案集上的指标变化、增加的延迟、增加的调用次数与 token 成本。
先试便宜的替代路径：条件触发、按会话缓存、用小模型跑改写，通常能压掉大半开销。
判据是「收益 × 受影响流量占比 ÷ 增加的延迟与成本」，再和其他候选优化排序。
延迟真的卡死时，把改写与首次检索并行发，两路名次用倒数排名融合，延迟从加法变成取最大值。
What happens when intent routing misclassifies, and how would you design the fallback?意图路由判错了会怎样？你会怎么设计兜底？
Common in ChinaCommon overseasIntermediate#intent-routing#fallback#observability
How to reason about it · think before answering
This tests whether you have thought about the direction of the error. A router is a classifier and classifiers misfire; "add more training data" is not a fallback design.
Break the errors down by direction — that is the backbone of the answer. Across three routes (direct answer, single-hop, multi-hop) the six confusions carry wildly asymmetric costs. Routing a retrieval-worthy question to a direct answer leaves the model with no material at all, so it fabricates: the most expensive error. Routing chit-chat to single-hop merely wastes one retrieval. Routing multi-hop to single-hop just yields an incomplete answer.
The conclusion follows: bias the fallback toward spending a little more, and default to single-hop retrieval whenever the classifier is unsure. Single-hop is the cheapest error to make, and it is recoverable — with partial material the model can still say it only found half the answer; with no material it can only invent one.
Add a runtime fallback, which beats better up-front classification: after a direct-answer routing, if the draft reply contains figures, amounts or dates that need a source, fall back to retrieval and answer again; after a single-hop routing, if no candidate clears the admission gate, escalate to multi-hop or abstain. Correcting the earlier decision with the later observation is the single most useful pattern in routing systems.
Mention observability: log every routing decision with the raw question, the label, and whether a fallback fired. Without that log you know neither how accurate the router is nor what to train the next version on.
Expect "when should you skip routing entirely?" When chit-chat is a small share of traffic and multi-hop questions are rare, the classification call costs more than it saves. In our 30-document lab the real gain from routing was not saved retrievals but the ability to give recognised multi-hop questions a larger context budget.
分析过程 · 先想清楚再作答
这题在考「有没有想过错误的方向」。路由是分类器，分类器一定会错；只答「多加训练数据提高准确率」的，等于没回答兜底怎么设计。
先把错误按方向拆开，这一步是整题的骨架：三条路（直接回答、单跳检索、多跳检索）两两误判，代价完全不对称。把该检索的判成直接回答，模型手里一点材料都没有，只能编，这是最贵的一种错；把闲聊判成单跳，只是白花一次检索；把多跳判成单跳，只是少查一轮、答得不全。
结论顺势就出来了：**兜底方向要偏向「多花一点钱」，判不出来一律退回单跳检索。** 单跳是三条路里错得最轻的一条，而且它的错误是可恢复的——材料不全模型还能说「资料里只查到一半」，材料为空它就只能编。
再补一层运行时兜底，比事前分类更管用：分类成直接回答之后，如果模型的回答里出现了具体数字、金额、日期这类需要出处的内容，就回退去检索一次再答；分类成单跳之后，如果检索侧一条都没过门槛，就升级走多跳或直接拒答。**用后一步的观测结果纠正前一步的判断**，这是路由系统最实用的一条设计。
还要提一句可观测性：路由的每一次判定都要落日志，带上原始问题、判定结果、后续是否发生了兜底升级。没有这份日志，你既不知道路由准不准，也没法攒出下一版的训练集。
可预期的追问是「什么时候干脆别做路由」。答：流量里闲聊占比很低、且多跳问题很少时，路由省下的钱还不够付分类调用的钱，这时候直接全部走单跳更划算——我们在 30 篇语料的实验里就看到，路由真正的收益并不在省检索，而在于认出多跳之后给它更高的上下文预算。
Key points
The three routes have asymmetric error costs: sending a retrieval-worthy question to a direct answer is the worst, while routing chit-chat to single-hop only wastes one retrieval.
Bias the fallback toward spending more: default to single-hop whenever the classifier is unsure, since that error is the mildest and is recoverable.
Add runtime fallbacks: re-retrieve if a direct answer contains figures that need a source; escalate or abstain if no single-hop candidate clears the gate.
Log every routing decision — raw question, label, whether a fallback fired — for both monitoring and the next training set.
When chit-chat and multi-hop are both rare, the classification call costs more than it saves; route everything to single-hop instead.
答题要点
三条路的误判代价不对称：把该检索的判成直接回答最贵（模型没材料只能编），把闲聊判成单跳只是白花一次检索。
兜底方向偏向多花钱：判不出来一律退回单跳检索，它是错得最轻且可恢复的一条路。
加运行时兜底：直接回答里出现需要出处的数字就补一次检索；单跳检索一条都没过门槛就升级或拒答。
每一次路由判定都落日志（原始问题、判定结果、是否触发兜底），既用于监控也用于攒下一版训练集。
闲聊与多跳占比都很低时，路由省的钱付不起分类调用，直接全走单跳更划算。

D11 Advanced Indexing: Parent-Child Documents, Summary Indexes, Contextual Retrieval, and the Trade-Offs of Tree Aggregation vs. Graph Retrieval

Parent-child indexing and contextual retrieval both patch the same problem — chunks losing their context. What actually distinguishes them?父子索引和上下文检索都在补『块被切碎』这个问题，它们的差别到底在哪？
Common in ChinaCommon overseasIntermediate#indexing#contextual-retrieval#chunking
How to reason about it · think before answering
The hinge is which half of the pipeline each one fixes. Answering 'one is a chunking trick, the other adds a prompt' just describes implementations; the interviewer wants to know where each acts.
Split the pipeline in two and ask separately: what does the retriever see, and what does the generator see. Parent-child changes the generation side — retrieval still runs on small chunks, but a hit is swapped for its parent. Contextual retrieval changes the retrieval side — the header exists so the chunk can be found at all, and the generator does not need it.
Conclusion: parent-child fixes 'found it but can't read it'; contextual retrieval fixes 'readable but never found'. Neither changes what the other changes, so they compose.
That difference also dictates which metric can see each one. Contextual retrieval moves rank, so recall and nDCG catch it. Parent-child moves 'is the evidence sufficient to answer', which a binary recall metric cannot see. Our 20-question set is already saturated at 100% on single-document questions, so parent-child comes out level with the baseline — that is the ruler failing, not the technique.
That difference yields a free optimization: since the header only serves retrieval, keep it out of the context window. Leaving it in pays rent on every single query. Flipping that one switch in our lab freed 36 tokens inside a 600-token budget with every metric unchanged.
The costs differ too. Parent-child costs index entries and a bigger context unit. Contextual retrieval costs one model call per chunk up front plus a permanently larger index. One is space; the other is time and space.
Expect the follow-up 'why not both'. Look at the failure logs first: are you mostly seeing incomplete evidence, or nothing retrieved at all? Without the matching failure mode, neither is worth its price.
分析过程 · 先想清楚再作答
这题的题眼是『补的是哪一半』。答成『一个是切块技巧、一个是加提示词』就是在描述实现，面试官想听的是它们各自作用在检索管道的哪一段。
拆的办法是把管道分成两段问：检索时看到什么、生成时看到什么。父子索引改的是**生成侧**——检索单位还是小块，只是命中之后把上下文单位换成大块；上下文检索改的是**检索侧**——块头拼进去是为了让这一块能被检索到，模型生成时并不需要它。
结论：父子索引解决『找到了但看不全』，上下文检索解决『看得全但找不到』。前者不改变谁被检索到，后者不改变模型看到多少。它们正交，可以叠加。
这个差别还决定了它们各自要用什么指标去量：上下文检索动的是名次，用召回率和 nDCG 量得到；父子索引动的是『材料够不够答』，召回率这种二值指标量不出来。我们那份 20 题评估集单文档档已经 100% 饱和，父子索引在表里跟基线持平——那不是它没用，是尺子量不了它。
顺着这条差异能推出一个立刻能用的优化：既然块头只服务检索，就不该进上下文。它进了上下文就是在每一次查询里白占预算，而且这笔钱是长期的。我们的实验里把这个开关一改，五列指标一个不变，600 token 的预算里多装进了 36 个 token。
代价也不同：父子索引的代价是索引条目变多、每次装进上下文的东西变大；上下文检索的代价是一次性要给每块调一次模型，加上索引 token 永久变大。前者是空间，后者是时间加空间。
可预期的追问是『那我全都上』。答案是先看失败案例：日志里是『材料不完整』多，还是『压根没检索到』多。没有对应的失败模式就不该上，这两个手法都不是免费的。
Key points
Parent-child acts on the generation side: retrieve small, swap in the parent for context. It fixes 'found but unreadable'.
Contextual retrieval acts on the retrieval side: the header makes the chunk findable. It fixes 'readable but never found'.
They are orthogonal and compose; keep the header in the index only, never in the context window.
Parent-child costs more index entries and a larger context unit; contextual retrieval costs one call per chunk plus a permanently larger index.
Pick based on the observed failure: incomplete evidence points to the former, zero retrieval to the latter.
答题要点
父子索引作用在生成侧：检索单位是小块，上下文单位换成父块，解决『找到了但看不全』。
上下文检索作用在检索侧：块头让块能被检索到，解决『看得全但找不到』。
两者正交可叠加；块头只该进索引不该进上下文，否则每次查询都在为它付钱。
父子索引的代价是索引条目与上下文单位变大；上下文检索的代价是一次性建索引调用加永久变大的索引。
选哪个看失败案例：材料不完整选前者，压根没检索到选后者。
Contextual retrieval needs one model call per chunk. How do you estimate that one-off cost, and what levers bring it down?上下文检索要给每个块调一次模型，这笔一次性成本怎么估？有哪些办法能压下来？
Common in ChinaCommon overseasDeep dive#contextual-retrieval#prompt-caching#cost
How to reason about it · think before answering
This checks whether you have actually done the arithmetic. Saying 'prompt caching makes it cheap' without knowing which line item it touches is a tell.
Split the bill first: one-off = per-chunk input + output + full re-embedding; per-query = the header read twice, once by the reranker and once in the context. Keep them separate, because they scale with completely different things.
The dominant term on the one-off side is how many times the same document is re-read. A doc split into n chunks is read n times. Prompt caching attacks exactly that: put the whole document first and mark it cacheable, pay a cache write once, then cache reads for the remaining n-1, typically an order of magnitude cheaper than input.
Order matters. Caching is prefix-matched, so the document must come first and the chunk after. Put the varying part first and the prefix changes every call — zero cache hits. This is the most common way people get it wrong.
Our measurement: 30 docs, 134 chunks. Without caching, 103017 input tokens; with caching, 17340 written plus 60137 read, cutting the one-off cost by roughly 29%. The finer the chunks, the bigger the saving, because re-reads multiply.
The counter-intuitive part is the useful part: the one-off cost amortizes below 10% of per-query cost after about 217 queries. The lasting bill is the extra tokens every query carries (we measured +12.3%). So the first lever is not cheaper index building — it is keeping the header out of the context, keeping it short, and not generating it for the whole corpus indiscriminately.
A bonus point: before spending any of it, confirm your evaluation setup can actually detect the benefit. In our offline harness the vector route contributed exactly zero unique answer documents, so it cannot answer whether headers help embeddings at all — an A/B run there hands you a wrong conclusion that looks numerically supported.
分析过程 · 先想清楚再作答
这题考的是你有没有真的算过账。只会说『用提示词缓存就便宜了』属于听过没做过——面试官会追问缓存到底省在哪一项上。
先把成本拆开：一次性 = 每块的输入 + 输出 + 全量 embedding；每次查询 = 块头在重排和上下文里各被读一遍。**这两笔要分开记**，因为它们随业务量的增长方式完全不同。
一次性那笔的主项是『同一篇文档被重复读了多少遍』。一篇切成 n 块就要读 n 遍，这是成本的大头。提示词缓存省的正是这一项：把整篇放在提示词最前面并标记为可缓存，第一块付一次缓存写入，后面 n-1 块只付缓存读取，而读取价通常比输入价低一个数量级。
顺序不能反：缓存按前缀匹配，整篇必须在前、块内容在后。把变化的块放前面，前缀次次都变，缓存一次都不会命中——这是最常见的翻车点。
我们的实测：30 篇、134 块，不开缓存输入 103017 token，开缓存后拆成写入 17340 加读取 60137，一次性成本降约 29%。**块切得越碎这个比例越高**，因为重复读的次数更多。
结论反直觉但很实用：一次性那笔是小钱，摊到 217 次查询就降到每次查询成本的一成以下；真正的长期账是每次查询多出来的那几十个 token（我们量到 +12.3%）。所以压成本的第一优先级不是压建索引，而是让块头别进上下文、别过长、别对全库无差别地生成。
最后一条是加分项：花这笔钱之前先确认你的评估环境**测得出**收益。我们的离线环境里向量路对召回的独立贡献实测为 0，所以它根本没法回答『块头对向量侧有没有用』——在这种环境里做的 A/B 会给你一个看起来有数字支撑的错误结论。
Key points
Split into one-off (per-chunk input/output plus re-embedding) and per-query (header read by both reranker and generator).
The one-off is dominated by re-reading each document n times; caching turns that into one write plus n-1 reads.
Caching is prefix-matched: the full document must come first, the chunk after, or you get zero hits.
Measured on 30 docs / 134 chunks, caching cut the one-off cost by about 29%, and finer chunks save more.
The lasting cost is per query: keep headers out of the context window, keep them short, and generate them selectively.
答题要点
把账拆成一次性（每块的输入输出 + 全量 embedding）和每次查询（块头在重排与上下文里各读一遍）两笔。
一次性的大头是同一篇被重复读 n 遍；提示词缓存把它压成一次写入加 n-1 次读取。
缓存按前缀匹配，整篇必须放在提示词最前面，块内容在后，顺序反了一次都不会命中。
实测 30 篇 134 块，一次性成本降约 29%，块越碎省得越多。
长期账在每次查询：块头别进上下文、控制长度、只对真正需要的文档生成。
What kind of question actually requires graph retrieval? Give one concrete case where it is justified and one where it is not.什么样的问题必须上图检索？给一个该上的具体例子和一个不该上的例子。
Common in ChinaCommon overseasDeep dive#graph-rag#multi-hop#cost
How to reason about it · think before answering
This one tests whether you reach for tools you don't need. If the answer is 'multi-hop questions need a graph', the interviewer knows you haven't shipped one — multi-hop is necessary, nowhere near sufficient.
Anchor the criterion on something observable: does the second required document share any lexical or semantic overlap with the query? If it does, ordinary hybrid retrieval will surface it and the hop is illusory. If it shares nothing, only a relation edge gets you there — that is graph territory.
Justified case: 'who must sign off on a production failover, and what is that person's name?' One doc says the platform lead must approve; another says who the platform lead is. The second shares not one term with the query. Across all five index structures we tested, it never once appeared in a 20-item candidate pool — rechunking, headers and parent backfill all failed.
Unjustified case: 'which process covers a capacity change, and how many working days ahead must the ticket be filed?' Also two documents, but both overlap the query lexically; hybrid retrieval ranked them second each, and one pass collected both. Building a graph for this buys a solved problem at several times the cost.
Then state the cost, which is what makes the answer sound operational: graph building is not one extraction call. Entities need disambiguation, relations need dedup, updates force recomputing affected subgraphs, and you now run a graph store and its update pipeline.
Expect 'what else could you do instead'. Hand multi-hop to agentic retrieval: let the model retrieve the intermediate entity first, then issue a second query with it. Near-zero build cost, paid back in latency and call count per query. Try that before you build a graph.
分析过程 · 先想清楚再作答
这题在考你会不会为了用而用。只要答案里出现『多跳问题就要上图检索』，面试官基本就知道你没落地过——多跳只是必要条件，远不是充分条件。
判据要落在一个可观察的现象上：**答案的第二篇文档和查询之间，有没有字面或语义上的重合**。有重合，普通的混合检索就能捞到它，多跳是假的；完全没有重合，只能靠一条关系边走过去，这才是图检索的领地。
该上的例子：问『生产库主备切换必须谁书面审批、这个人叫什么』。一篇写着须平台组组长审批，另一篇写着平台组组长是某人。第二篇跟查询一个词都不重合，我们在五种索引结构下测了一遍，它在 20 条候选池里一次都没出现过——换切法、加块头、父子回填全都无效。
不该上的例子：问『扩容要走哪个流程、最晚提前几个工作日提单』。同样跨两篇文档，但两篇都跟查询有明显字面重合，混合检索把它们分别排在第 2 名，一次检索就凑齐了。为它建图是拿几倍成本买一个已经解决的问题。
然后说代价，这一段决定了你像不像做过：建图不止一次抽取调用，实体要消歧、关系要去重、文档更新时受影响的子图要重算，还要多维护一套图存储和一套更新链路。
可预期的追问是『不上图检索还有什么办法』。答案是把多跳交给 Agentic 检索：让模型先查出中间实体，再拿这个实体发起第二次检索。它的一次性成本几乎为零，代价换成了每次查询的延迟与调用次数——先试这条，试不通再考虑建图。
Key points
The test is not 'is it multi-hop' but 'does the second document overlap the query at all' — only zero overlap earns a graph.
Justified: the approver question, where an intermediate entity is the only bridge and the second doc never enters the candidate pool.
Not justified: a multi-hop question whose documents both overlap the query — hybrid retrieval collects them in one pass.
Real graph cost is entity disambiguation, relation dedup, incremental subgraph recomputation and a whole extra store — not a single extraction call.
Try two-pass agentic retrieval first; build the graph only when that fails.
答题要点
判据不是『是不是多跳』，而是『第二篇文档跟查询有没有字面或语义重合』——没有重合才轮得到图检索。
该上：审批人那类问题，中间实体是唯一的桥，第二篇文档在候选池里一次都不出现。
不该上：两篇都跟查询有重合的多跳题，混合检索一次就能凑齐。
建图的真实成本是实体消歧、关系去重、增量重算和一套额外的图存储，不是一次抽取调用。
先试 Agentic 检索的两次查询，走不通再考虑建图。
You have built three different indexes over the same corpus. How do you decide which one a query goes to?同一份语料建了三套索引，检索时你怎么决定走哪一套？
Common in ChinaCommon overseasIntermediate#index-routing#evaluation#architecture
How to reason about it · think before answering
Whether this is an easy point or a lost one depends on whether you first ask 'do we actually need three?'. Jumping straight to routing accepts an unverified premise.
Step one is admitting the answer is usually 'none of them — use the default'. Across 30 documents we measured five index structures and every one landed at 93.8% recall, none beating the baseline. The only metric that moved was nDCG@10, which headers lifted from 0.6438 to 0.7218, while the two-stage summary index fell to 87.5%. Each structure patches one specific weakness; without that weakness it is pure overhead.
Step two is routing, and the criterion is not 'which index is more accurate' — that is an offline evaluation question, not something you know at request time. What you do have at request time is the shape of the question: detail-seeking, summarizing, or entity-chaining. Those map onto the chunk index, the tree-summary index and the graph index.
Implementation is a lightweight intent classifier — the same one from the previous day's intent routing, no need to invent another. Carry the decision as request metadata so you can replay it later.
Spell out the fallback: on a misclassification, fall back to the default index rather than fanning out across all three and fusing. Fan-out looks safe but multiplies latency and cost by the number of indexes, and the extra routes usually never make it into the context budget anyway.
Expect 'how do you know the classifier is right'. Log every routing decision and replay the golden set periodically: run each question through all three indexes and check whether the classifier picked the best-scoring one. It is a standing offline job that needs no human labelling.
分析过程 · 先想清楚再作答
这题是送分还是丢分，取决于你有没有先反问一句『真的需要三套吗』。上来就答路由策略的人，默认了一个没被验证的前提。
第一步是承认多数情况下答案是『都不走，走默认那套』。我们在 30 篇语料上把五种索引结构各测一遍，**召回率全部停在 93.8%，没有一种跑赢基线**；唯一动了的是 nDCG@10（块头把它从 0.6438 抬到 0.7218），而两段式的摘要索引还掉到了 87.5%。每种结构补的都是一个特定短板，你没有那个短板时它只带来成本。
第二步才是路由，而判据不是『哪套准』——那是离线评估该回答的问题，不是运行时能知道的。运行时能拿到的只有**问题的形状**：细节型（答案落在某一段）、概括型（要全库的一个概括）、多跳型（要跨实体串联）。按形状分流，正好对应块级索引、树状聚合索引、图索引。
实现上就是一个轻量意图分类器，跟前一天的意图路由是同一套东西，不必再造一个。分类结果作为元数据带进请求，方便事后拿评估集回看分错了多少。
兜底策略要说清楚：分类错了**回落到默认那一套**，不要并行全查一遍再融合。并行看着稳，实际上把延迟和成本按索引套数翻倍，而多出来的那两路大概率一条都进不了上下文预算。
可预期的追问是『怎么知道分类器分对了』。答案是把路由决策记进日志，定期拿标准答案集回放：对每个问题分别走三套索引，看分类器选的那套是不是指标最好的那套。这是一个能持续跑的离线作业，不需要人工标注。
Key points
First challenge the premise: all five index structures landed at the same 93.8% recall in our measurement, so an index without a matching weakness is pure cost.
At request time the usable signal is question shape — detail, summary, or entity-chaining — mapping to chunk, tree-summary and graph indexes.
Reuse the previous day's intent router for classification and record the routing decision as request metadata.
Fall back to the default index on misclassification instead of fanning out and fusing, which multiplies latency and cost.
Replay the golden set periodically to check whether the classifier picks the best-scoring index.
答题要点
先反问是否真需要三套：实测五种索引结构召回率全部持平在 93.8%，没有对应短板就是纯成本。
运行时的判据是问题的形状——细节型、概括型、多跳型，分别对应块级、树状摘要、图索引。
复用前一天的意图路由做分类，把路由决策记进请求元数据。
分类错了回落到默认索引，不要并行全查再融合——延迟和成本按套数翻倍。
用标准答案集定期回放，检验分类器选的那套是不是指标最好的那套。

D12 Agentic RAG: Turning Retrieval Into a Tool So the Model Decides Whether to Search, How Many Times, and Whether to Start Over

You are exposing retrieval to a model as a tool. How do you write the tool description, and what concrete failure modes appear when you write it badly?把检索包成一个工具交给模型，这个工具的描述该怎么写？写不好会导致哪些具体的错误行为？
Common in ChinaCommon overseasBasic#tool-design#agentic-rag#prompting
How to reason about it · think before answering
The discriminator is whether you can name concrete failure modes. Reciting 'the description should be clear' signals you have never shipped one.
Give the structure first: a usable description answers four things - what is and is not in the corpus, when the tool must be called, when it must not be called, and what shape the query string should take.
Attach a failure to each: no scope and the model treats it as a web search; no 'must call' and it answers policy questions from memory, convincingly; no 'must not call' and greetings or translations each burn a retrieval; no query shape and the model pastes the raw user sentence in, dragging interrogative words into the index.
The query-shape line is the cheapest win: one sentence saying 'keyword phrase, no question words' beats ten heuristics for query cleaning on the retrieval side.
Production angle: optional filter parameters such as department need an explicit 'only set this when you are certain'. Models like to fill optional fields, and a wrong filter hides the correct answer while the logs only show 'no results'.
Expected follow-up: how do you verify the description works? Run a negative suite - small talk, translation, arithmetic, follow-ups already answered in the conversation - and assert the tool was not called. That regression is automatable.
分析过程 · 先想清楚再作答
这题的题眼是「具体的错误行为」。只会背「描述要写清楚工具的用途」的，一句话就暴露了没上过线——面试官想听的是描述里少一句话，线上就多一类工单。
先给结构：一段合格的工具描述要回答四件事——库里有什么和没有什么、什么时候必须用、什么时候不要用、查询串写成什么形状。四条各对应一类事故，逐条挂钩着说最有说服力。
逐条挂钩：不写范围，模型拿它当搜索引擎，问天气也去查；不写「必须用」，涉及公司制度的问题被模型凭记忆编答案，而且编得非常像真的；不写「不要用」，闲聊和翻译都触发一次无谓检索，成本和延迟白涨；不写查询形状，模型把用户整句问话塞进 query，「叫什么名字」这种疑问词进了检索，纯噪声。
最后一条最值钱也最容易漏：在描述里加一句「写成关键词短语，不要带疑问词」，比在检索侧做十种查询清洗都管用——问题在源头，就在源头修。
补一个生产视角：参数里的过滤字段（比如部门）要写明「只在确定时才填」。模型倾向于把可选参数填满，填错一个部门就把正确答案挡在库外，而这种错误在日志里看不出来，表现是「检索没结果」。
可预期的追问是「怎么验证描述写对了」。答案是拿一批负样本跑：闲聊、翻译、算术、以及答案已在对话里的追问，看模型有没有多调一次工具；这类回归是能自动化的。
Key points
The description is a prompt for the model, not a code comment: scope, when to call, when not to call, query shape.
Missing scope turns it into a web search; missing 'must call' produces confident answers from memory.
Missing 'do not call' makes small talk trigger retrieval, paying cost and latency for nothing.
Stating 'keyword phrase, no question words' fixes query pollution at the source.
Optional filters need 'only set when certain' - a wrong filter silently hides the right answer.
Regression-test with a negative suite and assert the tool was not invoked.
答题要点
描述是写给模型看的提示词，不是注释；四段式：范围、什么时候用、什么时候不用、查询写成什么形状。
不写范围会被当成搜索引擎；不写「必须用」会导致凭记忆编答案。
不写「不要用」会让闲聊也触发检索，成本和延迟白涨。
写明查询要用关键词短语、不带疑问词，比在检索侧清洗查询更根本。
可选过滤参数要写「只在确定时才填」，填错会静默地把正确答案挡在外面。
用一批负样本（闲聊、翻译、算术）做回归，断言工具没有被调用。
Self-reflective retrieval rewrites the query and retries. How do you guarantee it terminates instead of spinning on the same query forever?自反思式检索会反复改写查询重试。你怎么保证它一定会停下来，而不是在同一个查询上原地打转？
Common in ChinaCommon overseasIntermediate#agentic-rag#self-reflection#reliability
How to reason about it · think before answering
This checks whether you have actually run such a loop. 'Set a max iteration count' is half an answer: it stops one failure mode and lets two others through.
Split runaway behaviour into three shapes and give each its own brake. Progress that never completes is capped by max rounds. Per-round budgets that pass individually but blow up in aggregate need a cumulative token budget - four rounds of 600 tokens each never trips a per-round check yet quadruples what reaches the model. Spinning in place needs duplicate-query detection.
Two implementation details prove you have written it: the duplicate check belongs before the retrieval call, otherwise you pay for a call to learn you are looping; and queries must be normalized to a set of terms, or 'failover approval' and 'approval failover' count as two distinct queries and the loop keeps turning.
Say what happens after it stops: stop reasons must be recorded as distinct categories - satisfied, gave up, hit round cap, hit token budget, duplicate query. Collapsing them into 'loop finished' hides how often the system simply surrendered.
An easy miss: installing a brake is not testing it. If the default token budget sits far above real usage it never fires, which is the same as not having one. Every brake needs a case that trips it.
Expected follow-up: what if the model says 'not enough' when it actually is? Make the assessment structured - which elements are covered, which are missing - and treat an empty missing list as sufficient, so the decision is auditable rather than a bare boolean.
分析过程 · 先想清楚再作答
这题在考「有没有真让循环跑过」。只答「设一个最大轮数」的能拿一半分，因为最大轮数只拦住了一类失控，剩下两类照样漏出去。
怎么拆：把失控分成三种形态，每种配一道闸。一是「每轮都在推进但永远推进不完」，用最大轮数拦；二是「每轮都不超标但累计爆掉」，用累计 token 预算拦——四轮各读 600 token 没有一轮超标，可送进模型的材料已经是单轮的四倍；三是「原地打转」，用重复查询检测拦。
重复查询检测有两个实现细节，答出来就说明真写过：一是要放在检索之前，否则要白花一次调用才发现自己在转圈；二是判重要对查询做归一化，只看词的集合，否则「主备切换审批」和「审批主备切换」会被当成两个不同的查询，圈照转不误。
还要说清停下来之后怎么办：停止原因必须分类记录，「查够了」「主动认输」「撞到轮数」「撞到预算」「原地打转」是五种不同的结局。把它们混成一个「循环结束」，你就永远看不见系统在多大比例的问题上其实是放弃了。
一个容易被忽略的点：闸门装了不等于验过。默认预算如果比实际用量高一大截，跑多少遍都踩不响它，等于没装。每一道闸都要构造一个用例把它踩响，这是验收的一部分。
可预期的追问是「模型自己说不够，但其实已经够了怎么办」。答案是自评要给结构化输出（覆盖了哪些要素、缺哪些），缺失项为空却仍判不够时按「够了」处理——让判断可审计，而不是信一个布尔值。
Key points
Three brakes, none optional: max rounds, cumulative token budget, duplicate-query detection.
The cumulative budget catches rounds that each pass but blow up together - the round cap cannot see that.
Check for duplicates before retrieving, and normalize the query to a term set before comparing.
Record stop reasons as distinct categories rather than one 'finished' bucket.
Every brake needs a case that actually trips it; an untested brake is no brake.
Have the assessor emit covered and missing elements so 'not enough' is auditable.
答题要点
三道闸缺一不可：最大轮数、累计 token 预算、重复查询检测。
累计预算拦的是「每轮都不超但加起来爆掉」，轮数闸看不见这件事。
重复查询检测要放在检索之前，且查询要归一化成词的集合再判重。
停止原因分类记录：查够了、主动认输、撞轮数、撞预算、原地打转是五种结局。
每一道闸都要构造用例踩响，装了没验过等于没装。
自评输出结构化的覆盖与缺失项，让「不够」这个判断可审计。
In multi-hop retrieval a wrong first hop poisons every hop after it. How would you design for that?多跳检索里第一跳查错了，后面全跟着错。你会怎么设计容错？
Common in ChinaCommon overseasDeep dive#multi-hop#error-propagation#agentic-rag
How to reason about it · think before answering
This is about error propagation. 'Add a retry' is not enough - retries help when one path fails, but the multi-hop problem is walking confidently down the wrong path.
Separate two failure kinds first, because the fixes are opposite. Either the answer document never entered the candidate pool - no amount of loosening helps, only a new query term does, which is the multi-hop path - or it was retrieved and then dropped by your own admission threshold, where extra hops are useless and only relaxing the gate recovers it. Coverage plus the answer slot tells them apart: low coverage means wrong direction (broaden), high coverage with an empty slot means halfway there (hop).
Then give the mechanism. Do not let the model freestyle the next query: pick a bridge phrase from the sentence that best matches the question - a concrete noun the question never mentioned that also appears in another document. 'Not in the question' makes it new information; 'appears elsewhere' guarantees there is somewhere to hop to.
Fault tolerance has three layers: keep the earlier hop's material, so a bad second hop does not destroy the evidence you already had; trace each hop separately so you can locate where it went wrong; and surrender explicitly when there is no lead left, handing 'insufficient evidence' to the generation-side refusal.
The overlooked trap is worth points: the hop succeeds but the metric does not move. The second hop really did retrieve the target document, yet if you merge both hops' candidates and pack by score, the first hop's higher lexical overlap fills the budget and the target never enters the context. Allocate the context budget round-robin across hops - that is where multi-hop gains are actually realized.
Expected follow-up: how do you know the first hop was wrong? From the structured self-assessment, not from the final answer. By the time the answer is wrong the chain is three hops deep and much more expensive to debug.
分析过程 · 先想清楚再作答
这题考的是错误传播意识。只答「加个重试」是不够的——重试只在「同一条路走不通」时有用，而多跳的问题是走上了错误的路还越走越远。
先把两类失败分开，这是整题的骨架：一类是根本没捞到（答案文档在候选池里一次都没出现，放宽门槛毫无用处，只能靠新的查询词重查，也就是多跳），一类是捞到了却被自己的过滤器扔了（排在第二名但没过准入门槛，这一类跳多少跳都没用，只能降级放宽门槛重判）。判据是要素覆盖率加答案槽位：覆盖率低是方向错了走放宽，覆盖率高但槽位空是只查到半路走多跳。判错类型，容错就完全用反了。
然后给具体机制。判断下一跳查什么，不能凭模型自由发挥，要有可解释的判据：从最贴题的那一句里挑出问题没提过、且在别的文档里也出现过的具体名词当作桥接短语。「问题没提过」保证它是新信息，「别的文档里也有」保证真的有下一跳可跳——只在这一篇里出现的短语，查了只会把同一篇再捞回来。
结论层面，容错有三层：不要丢掉上一跳的材料（第二跳查错了，第一跳的证据还在）；每一跳独立记录轨迹，事后能定位是哪一跳歪的；追不动时主动认输，把「材料不足」交给生成侧的拒答，而不是硬凑一个答案。
有一个非常容易被忽略的坑，说出来会加分：跳成功了，命中却没变。第二跳确实把目标文档检索回来了，但如果把两跳的候选混在一起按名次装上下文，第一跳的材料字面重合度更高，会把预算占满，目标文档根本挤不进去。上下文预算必须按跳轮转分配——多跳的收益是在这一步兑现的，不是在检索那一步。
可预期的追问是「怎么知道第一跳错了」。答案是靠自评的结构化输出，而不是靠最终答案对不对；等到答案错了再回头找，链路已经断了三跳，定位成本高得多。
Key points
Classify first: never retrieved needs a new query term (a hop); retrieved-then-filtered needs a relaxed gate. The fixes are opposite.
Derive the next query from a bridge phrase - a concrete noun absent from the question that also appears in another document.
Keep the previous hop's material so a failed hop does not discard existing evidence.
Trace every hop separately so you can pinpoint which one drifted.
Allocate context budget round-robin across hops, or a successful hop still fails to change the metric.
Surrender explicitly when no lead remains and hand it to the generation-side refusal.
答题要点
先分类：根本没捞到只能靠多跳换查询词，捞到了被门槛扔了只能靠降级放宽，两者修法相反。
下一跳的查询用桥接短语：问题没提过、且别的文档里也出现过的具体名词。
保留上一跳的材料，第二跳失败时第一跳的证据仍在。
每一跳独立记轨迹，能定位是哪一跳歪的。
上下文预算按跳轮转分配，否则跳成功了命中也不会变。
追不动时主动认输，把材料不足交给生成侧拒答，不硬凑答案。
When would you refuse to make a RAG system agentic, and what data would you use to convince your team?什么情况下你会拒绝把一个 RAG 系统做成 Agentic 的？拿什么数据说服你的团队？
Common in ChinaCommon overseasIntermediate#agentic-rag#cost#engineering-judgement
How to reason about it · think before answering
This tests engineering judgement and whether you can do arithmetic. Anyone who says 'agentic is more advanced so we should ship it' is out. The interviewer wants you to name the cost and draw the boundary with numbers.
Decompose it: identify which question types actually benefit, then check how much of your traffic they represent. Agentic gains concentrate in multi-hop questions and retrieval retries; single-document questions are answered by one lookup and every extra round is waste.
So the criterion is the evaluation set, not intuition. On a 20-item set we measured multi-hop recall going from 75% to 100% while overall answerable recall moved only from 93.8% to 100%, at the cost of average retrieval calls going from 1 to 1.75 plus the same number of assessment calls - you pay for 100% of traffic so that 5% of it improves.
Three clear refusals: latency-sensitive surfaces, where each round adds a retrieval plus a model round trip and roughly doubles time to first token; fixed question patterns, where nine in ten questions are single-document and the gain is near zero; and tight cost budgets, where a real model is less disciplined than an offline stand-in and the variance, not the mean, is what breaks your capacity plan.
Finish with the alternative: route. Use one cheap check to decide whether a question looks multi-hop, and only then enter the loop. Nine tenths take a single retrieval, one tenth loops, and the economics change completely. Looping is a capability, not a default.
Expected follow-up: how do you know which questions look multi-hop? Mine the eval set and production logs for patterns - two facts requested in one sentence, or a question about the person behind a role - start with rules, and reach for a small classifier only when rules stop working.
分析过程 · 先想清楚再作答
这题在考工程判断力，也在考你会不会算账。凡是答「Agentic 更先进所以要上」的，直接出局；面试官想听的是你能主动说出它的代价，并且用数字划出适用边界。
怎么拆：先承认收益来自哪一类问题，再看这类问题在你的流量里占多大比例。Agentic 的收益几乎全部集中在多跳和检索失败重试上，单文档可答的问题一次检索就够了，多查一轮纯属浪费。
所以判据不是感觉，是评估集：跑一遍，看 multi 那一档占多少题、涨了多少个点，再对照总调用次数涨了多少倍。在一份 20 题的集合上，我们量到的是多跳召回从 75% 涨到 100%，可答题整体只从 93.8% 涨到 100%，代价是平均检索调用从 1 次涨到 1.75 次、外加同样次数的自评调用——为 100% 的问题付钱，只有 5% 的问题拿到好处。
三类明确不上：延迟敏感（每多一轮就是一次检索加一次模型往返，首字延迟拉长一到两倍）；问题模式固定（九成是单文档可答，收益接近零）；成本吃紧（真实模型不像离线替身那样老实，成本方差比均值更难受，按均值做的容量规划会在长尾上被打穿）。
给出替代方案才算完整：分流。先用一次便宜的判断看这一问像不像多跳，像才进循环，不像走固定流程。九成走一次检索、一成走循环，账完全不一样。这也说明循环是一种能力，不是默认值。
可预期的追问是「那你怎么知道哪些问题像多跳」。答案是从评估集和线上日志里找模式（问句里同时问了两个事实、问的是某个角色背后的人），先用规则跑，跑不动再上小模型分类——顺序不要反。
Key points
Gains concentrate in multi-hop and retry cases; single-document questions gain almost nothing.
Settle it with the evaluation set: multi-hop delta against the multiplier on total calls.
One measured set: multi-hop recall 75% to 100%, overall 93.8% to 100%, retrieval calls 1 to 1.75 plus the same number of assessment calls.
Refuse when latency-sensitive, when question patterns are fixed, or when cost is tight - variance hurts more than the mean.
Route instead: a cheap check up front, and only multi-hop-looking questions enter the loop.
Looping is a capability, not a default.
答题要点
收益集中在多跳与检索失败重试，单文档可答的问题上收益接近零。
用评估集算账：multi 档涨了多少点，对照总调用次数涨了多少倍。
实测过的一组数字：多跳召回 75% 到 100%，整体 93.8% 到 100%，检索调用 1 次到 1.75 次外加等量自评调用。
三类不上：延迟敏感、问题模式固定、成本吃紧（方差比均值更难受）。
替代方案是分流：便宜的判断先过滤，像多跳才进循环。
循环是一种能力，不是默认值。

D13 Going to Production: Incremental Sync and Deduplication, Permission-Based Filtering, Cache Layering, Tracing, and the Cost-Latency Ledger

After a document changes, how do you recompute only the affected chunks? And how do you guarantee a deleted document really disappears from the index?文档更新之后，你怎么做到只重算受影响的块？被删掉的文档又怎么保证一定从索引里消失？
Common in ChinaCommon overseasIntermediate#incremental-sync#content-hash#index-maintenance
How to reason about it · think before answering
There are two halves here and the second one separates candidates. Almost everyone can say 'hash it and compare'; the score comes from bringing up deletion yourself, because it is the one asymmetric case in the whole mechanism.
Give the skeleton first: a three-way reconciliation between the full set from the source and the full set in the index. In source but not indexed is an add; in both but with different content hashes is a modify; indexed but absent from the source is a delete. A modify must replace the document wholesale, deleting old chunks before writing new ones, otherwise a shortened document leaves a tail behind in the index.
Then the fingerprint itself, which is where points are won: sha256 truncated, but normalize line endings and trim before hashing. The same file uploaded from Windows and from macOS differs byte-wise but not in content; skip normalization and every re-upload counts as a change, which is a full rebuild in disguise. It never raises an error, it only shows up on the bill.
The key insight in the second half: a deletion is not an event, it is an absence. Change feeds tell you what changed; nobody ever sends 'I no longer exist'. So deletion detection has to run in the opposite direction — walk the index and find ids the source no longer has. A synchronizer that only listens to change events will wait forever.
At the storage layer, cascade the foreign keys across documents, chunks and embeddings so deleting a document is a single statement and the database does the rest. Hand-written three-step deletes eventually miss one, and the one they miss is a ghost in the index. Close with a verifiable invariant: chunk count must equal embedding count, and a mismatch means orphans.
Expected follow-up: what if the source system itself is unreliable and a pull comes back incomplete? Make pull completeness a precondition for deletion: on a partial pull, apply adds and modifies only, or one failed fetch wipes half your index. Also soft-delete with a retention window so a mistake is recoverable.
分析过程 · 先想清楚再作答
这题有两半，区分度全在后半。前半几乎人人答得出「算个哈希比一比」，能不能拿到分取决于你有没有主动讲删除——那是同一套机制里唯一不对称的一种变更。
先给增量的骨架：拿来源的全集和索引的全集做三向对账。来源有、索引没有是新增；两边都有但内容指纹不同是修改；索引有、来源没有是删除。修改的处理是整篇替换，先删旧块再写新块，不能只追加——不然改短了的文档会在索引里留下一截尾巴。
接着讲指纹本身，这是给分点：sha256 取前若干位，但**算之前必须先做换行归一化再去首尾空白**。同一份文件从 Windows 传一次、从 Mac 传一次，字节不同内容相同，不归一化就每次都判成变了，等于天天在做全量重建。这个 bug 不报错，只体现在账单上。
然后是删除这一半的关键判断：**删除不是一个事件，是一个缺席**。文件变动类的通知只告诉你哪些东西变了，永远不会有人发一条「我不存在了」。所以删除检测必须反着来——遍历索引，找出来源里已经没有的 id。只监听变更事件的同步器永远等不到这条消息。
落到存储上：文档、块、向量三张表用外键级联删除，删文档只写一条语句，剩下的交给数据库。手写三条删除的版本迟早会漏掉一条，而漏掉的那条就是索引里的幽灵。收尾时报一个可验证的指标：块数与向量数必须相等，不等就说明有孤儿。
可预期的追问：来源系统本身就不可靠、拉不全怎么办？那就把「本次拉取是否完整」当成删除检测的前置条件——拉取不完整时只做新增和修改，不做删除，否则一次拉取失败会把半个索引清空。另外给删除加软删标记和保留期，误删还能回滚。
Key points
Three-way reconciliation covering adds, modifies and deletes; a modify replaces the whole document, old chunks first.
Normalize line endings and trim before hashing, or cross-platform re-uploads look like edits and you are doing a full rebuild every night.
Deletion is an absence, not an event: walk the index for ids the source no longer has instead of waiting on a change feed.
Cascade deletes from documents to chunks to embeddings so one statement suffices; assert chunk count equals embedding count to catch orphans.
On an incomplete pull, apply adds and modifies only, and soft-delete with a retention window so mistakes are reversible.
答题要点
三向对账：新增、修改、删除，缺一不可；修改是整篇替换，先删旧块再写新块。
内容指纹算之前必须先做换行归一化再 trim，否则跨系统重传会被误判为修改，等于天天全量重建。
删除是缺席不是事件，必须反过来遍历索引找出来源里已消失的 id，不能只监听变更通知。
文档、块、向量用外键级联删除，删文档只写一条语句；用「块数等于向量数」当可验证的收尾指标。
来源拉取不完整时只做新增与修改、跳过删除，并给删除加软删与保留期以便回滚。
Why can't access control be applied at the generation step? What exactly leaks if you put it there?为什么权限过滤不能放在生成阶段做？放在那里会泄露什么？
Common in ChinaCommon overseasDeep dive#access-control#filter-pushdown#multi-tenancy
How to reason about it · think before answering
This checks whether you think about RAG as a system. 'Because it's insecure' scores nothing; the interviewer wants what specifically leaks, and what else goes wrong besides the leak.
Anchor the position with an image: the archivist spreads every file on the table, you pick nine, and only then does he pull three back saying you may not read those. You have already seen the titles. Filtering at generation time is that gesture.
Then split the consequences, and note the second one is what shows engineering experience. First, exposure: the unauthorized documents were retrieved, ranked, read into process memory, and almost certainly written to retrieval logs and traces, even if none of their text reaches the answer. Second, dilution: you take the top 8, three are off-limits, the user gets five, and the legitimate results ranked ninth and tenth never get promoted. The user experiences 'it can't find anything' while your logs show a perfectly normal retrieval.
State the fix: put the permission predicate in the same query as the ordering and the LIMIT, so the database prunes rows before ranking and unauthorized vectors are never compared. Cover both shapes: row-level filtering is one index plus a predicate; index isolation is a separate index per boundary.
Give the selection criterion: the number and stability of the isolation boundaries. A handful of departments that rarely change makes isolation worthwhile; tens of thousands of per-user private document sets leave you with row-level filtering, because that many indexes is unmanageable. Add the shared-index side effect: a large tenant degrades everyone else's retrieval quality because candidate slots are shared.
Expected follow-up: what about caching? It is the same bug's second crime scene. The answer cache key must include the permission scope, or one user's answer will be served to another, and that leak leaves no trace in the retrieval log at all.
分析过程 · 先想清楚再作答
这题在考你有没有真的把 RAG 当系统看。答成「因为不安全」拿不到分，面试官要的是「具体泄露了什么」和「除了泄露还有什么后果」两件事。
先用一个画面把位置说清楚：档案管理员先把全部档案摊在桌上让你挑，你挑完他再抽走三份说这些不能看——你已经看见标题了。在生成阶段过滤就是这个动作。
然后拆后果，两条，第二条更能显出做过工程：一是**泄露面**，越权文档已经进过检索、参与过排序、被进程读进过内存、大概率写进了检索日志和链路追踪，哪怕最终答案里没有它的内容；二是**结果被稀释**，取前 8 条里有 3 条不该看，筛掉只剩 5 条，而本该补位的第 9、10 名合法结果永远没机会上来——用户体感是「查不到」，你的日志里却是一次正常检索。
给正确做法：把权限谓词和排序、LIMIT 写进同一条查询，数据库先裁行再排序取前 k，越权的行一次都没被比较过。两种落法要都讲：行级过滤是一份索引加一个谓词，索引隔离是按边界各建各的索引。
选型判据要给出来：看隔离边界的数量和稳定性。部门这种个位数且几乎不变的边界，隔离划算；几万个用户各自的私有文档就只能行级过滤，否则运维扛不住。补一句共用索引的副作用——数据量大的租户会拖慢别人的检索质量，因为候选名额是共享的。
可预期的追问：缓存怎么办？这是同一个问题的第二现场——答案缓存的 key 里必须带上权限范围，否则一个用户的答案会被另一个用户命中，而且这条泄露路径连检索日志都不会留下痕迹。
Key points
Filtering at generation time means unauthorized documents were already retrieved, ranked, held in memory and written to logs and traces; the exposure is far wider than 'did the text reach the answer'.
The second consequence is dilution: filtered-out slots are not backfilled, so users see 'nothing found' while the log shows a normal retrieval.
The fix is to put the permission predicate in the same statement as ordering and LIMIT so the database prunes before ranking.
Choose between row-level filtering and index isolation by the count and stability of the boundaries; a shared index lets a large tenant crowd out a small one's candidate slots.
Caching is the same bug's second crime scene: the answer cache key must carry the permission scope or answers leak across users without a trace.
答题要点
在生成阶段过滤时，越权文档已经被检索、排序、读进内存并写进日志与追踪，泄露面比「答案里有没有」大得多。
第二个后果是结果被稀释：筛掉之后名额空着不补，用户体感是查不到，日志里却是一次正常检索。
正确做法是把权限谓词和排序、LIMIT 写进同一条查询，让数据库先裁行再排序取前 k。
行级过滤与索引隔离的选型判据是隔离边界的数量与稳定性；共用索引时大租户会挤占小租户的候选名额。
缓存是同一个漏洞的第二现场：答案缓存的 key 必须包含权限范围，否则会跨用户串答案且不留痕迹。
What can be cached in a RAG system, and what are the invalidation conditions for each?RAG 系统里有哪些东西可以缓存？各自的失效条件是什么？
Common in ChinaCommon overseasIntermediate#caching#invalidation#cost-optimization
How to reason about it · think before answering
This looks like a giveaway and is actually a filter. 'Cache the question and answer' earns a third of the credit; the interviewer is waiting for the layering and the per-layer invalidation rules.
Lead with a transferable rule: 'when must this be invalidated' is the same question as 'is that thing part of the key'. Leave something out of the key and changes to it will never invalidate the entry. With that rule the three layers derive themselves.
Then go layer by layer. The answer layer maps a question to a final answer; its key needs the question, the permission scope, the index version, and the model plus prompt version. The retrieval layer maps a query to a hit list; its key needs the question, scope, topK, index version and embedding backend, but not the generation model. The embedding layer maps text to a vector; its key is just the text and the backend.
Emphasize the counterintuitive part of the embedding layer: it is content-addressed, so the index version must not be in its key. Put it there and a single sync invalidates tens of thousands of vectors, which is exactly the full rebuild you added caching to avoid. This is the one layer that can live a long time, even on disk.
Offer a concrete invalidation mechanism: version numbers rather than targeted deletion. Bump an index version whenever a sync actually changes something and old keys simply stop being computed. Targeted deletion would require enumerating which questions a change affected, and that list cannot be produced.
Expected follow-up: can you give a real 'should have expired but didn't' case? Yes: an answer cache keyed only on the question. A document's limit changes from 200 MB to 500 MB, the index is updated, and the same question still returns 200 MB. Nothing errors; the log shows a clean cache hit. The same key also serves one department's answer to a user from another.
分析过程 · 先想清楚再作答
这题看起来是送分题，实际是筛人题。答成「把问答结果缓存起来」只拿到三分之一，面试官等着听的是「分几层」和「各自什么时候失效」。
先给一条能迁移到别的题上的判断依据：**「什么时候必须失效」这个问题，等价于「key 里有没有把那样东西算进去」。** key 少放一样，那样东西变了缓存就不会失效。有了这条，三层的答案自己就长出来了。
然后逐层给：答案层缓存问题到最终答案，key 要有问题、权限范围、索引版本、模型与提示词版本；检索层缓存检索式到命中块列表，key 要有问题、权限范围、topK、索引版本、向量后端，但不需要模型；向量层缓存文本到向量，key 只有文本和向量后端。
重点讲向量层的反直觉之处：它是**内容寻址**的，文本没变、模型没变，向量就不会变，所以**不能把索引版本放进它的 key**。放进去的话一次同步就作废几万条向量，正好绕回全量重建——你加缓存想省的那笔钱又花回去了。这一层可以放很久甚至持久化。
给一个具体的失效手法：用**索引版本号**而不是精确删除。同步只要真的改动了索引就把版本号加一，旧 key 再也算不出来，自然没人读得到。精确删除要求你能列出「这次改动影响了哪些问题」，而那是列不出来的。
可预期的追问：能举一个「该失效却没失效」的真实例子吗？答：答案缓存的 key 只放了问题本身，文档里的上限从 200 MB 改成 500 MB、索引已经更新，再问同一个问题仍然返回 200 MB。它不报错，日志上是一次漂亮的缓存命中；同一个 key 还会让另一个部门的用户直接命中别人的答案。
Key points
Three layers — answer, retrieval, embedding — with lifetimes orders of magnitude apart; treating them as one thing is the mistake.
The rule is that 'when must it expire' equals 'is it in the key'; anything left out of the key can never invalidate the entry.
The answer key carries question, permission scope, index version, model and prompt version; the retrieval key drops the model and adds topK and the embedding backend.
The embedding layer is content-addressed and keyed only on text plus backend; adding an index version turns every sync back into a full rebuild.
Version-based invalidation beats targeted deletion because you cannot enumerate which questions a given change affected.
答题要点
分三层：答案、检索、向量，三者的寿命差着数量级，不能当成一件事。
判断依据是「什么时候必须失效」等价于「key 里有没有算进那样东西」，key 少一样就永远失效不了。
答案层 key 要有问题、权限范围、索引版本、模型与提示词版本；检索层去掉模型、加上 topK 与向量后端。
向量层是内容寻址的，key 只有文本与后端；把索引版本放进去会让每次同步都退化成全量重建。
用索引版本号做失效比精确删除可靠，因为「这次改动影响了哪些问题」根本列不出来。
You need to switch embedding models. How do you migrate a live system without downtime and without losing recall?要换一个 embedding 模型，线上系统怎么迁移才能不停机也不掉召回？
Common in ChinaCommon overseasDeep dive#embedding-migration#zero-downtime#rollout
How to reason about it · think before answering
The crux is why you cannot swap in place. Jumping straight to the steps without establishing that reads like reciting a runbook.
Set up the premise: vectors from different models are not comparable. Dimensions may differ, and even at equal dimensions the coordinate spaces are unrelated, so encoding the query with the new model and comparing against documents encoded with the old one yields noise. Switching models therefore means re-embedding the entire corpus.
Then the four steps: add a nullable second vector column; backfill it with a background job while the old column is untouched and still serves live traffic; canary a slice of traffic onto the new column while running the golden set against both columns to compare recall and faithfulness; cut over fully once the numbers hold, and drop the old column only after a week or two of observation.
Name the payoff explicitly, because this is where the points are: the value of the whole procedure is the rollback cost. Cutover is a config change naming which column to read, so reverting takes a second rather than re-running an eight-hour rebuild. A migration plan with no rollback path is not a plan.
Add two engineering details: build the approximate-nearest-neighbour index on the new column after the backfill, not during it, since concurrent building is slow and prone to locking; and make the backfill resumable and rate-limited, or it will exhaust the embedding API quota and drag live queries down with it.
Expected follow-up: how do you prove the new model is actually better? Not from an offline metric alone — run an A/B on the same golden set with identical retrieval parameters and report four numbers: recall, faithfulness, latency and cost. A conclusion resting on the first number only does not hold. Note also that switching models is the one moment when the embedding cache genuinely must be invalidated.
分析过程 · 先想清楚再作答
这题的题眼是「为什么不能就地换」。没有先说清这一点就直接讲步骤，会显得是在背流程。
先给前提：不同模型的向量之间**没有可比性**。维度可能不同，即使维度相同坐标系也完全不是一回事，用新模型编码问题去和旧模型编码的文档比距离，算出来的相似度是纯噪声。所以「换模型」实质上等于「把整个知识库重新向量化一遍」。
然后给四步：加一列新向量、允许为空；后台任务慢慢回填新列，旧列一个字节不动，线上仍走旧列；小流量灰度到新列，同时用标准答案集在两列上各跑一遍比召回率与忠实度；数字站得住再全量切换，旧列观察一两周后才删。
把这套流程的价值点破，这是给分点：**它的价值全在回滚成本上**。切换只是改一个配置项「走哪一列」，出问题时切回去是一秒钟的事，而不是重跑一遍八小时的重建任务。凡是拿不出回滚路径的迁移方案都不算方案。
补两个工程细节：新列的近似最近邻索引要在回填完之后再建，边写边建又慢又容易锁表；回填要能断点续传并限速，否则会把 embedding 接口的配额打满，把线上查询一起拖垮。
可预期的追问：怎么证明新模型确实更好？答：不能只看离线指标涨没涨，要在同一份标准答案集、同一套检索参数下跑 A/B，报召回率、忠实度、延迟、花费四笔账；只报第一笔的结论不成立。另外注意换模型会让缓存里的向量全部作废，那是这次迁移唯一该作废向量缓存的时刻。
Key points
Vectors from different models are not comparable, so a model switch is equivalent to re-embedding the entire corpus.
Four steps: add a nullable second vector column, backfill in the background, canary with the golden set scored on both columns, then cut over once the numbers hold.
The whole value lies in rollback cost: cutover is a config change, so reverting takes a second instead of another full rebuild.
Build the ANN index on the new column after the backfill; make the backfill resumable and rate-limited so it does not exhaust the embedding quota and stall live queries.
Validate with an A/B on one golden set reporting recall, faithfulness, latency and cost; a model switch is also the only time the embedding cache truly must be invalidated.
答题要点
不同模型的向量之间没有可比性，所以换模型等价于把整个知识库重新向量化一遍。
四步：加一列可空的新向量、后台回填、小流量灰度并用标准答案集在两列上对比、数字站得住再全量切换。
这套流程的价值全在回滚成本上：切换是改一个配置项，回滚是一秒钟的事而不是重跑一次重建。
新列的近似最近邻索引在回填完成后再建；回填要可断点续传并限速，别把接口配额打满拖垮线上查询。
验证要在同一份标准答案集上跑 A/B，同时报召回率、忠实度、延迟与花费四笔账；换模型也是唯一该作废向量缓存的时刻。

D14 Capstone Project and Retrospective: A Multi-Tenant Enterprise Knowledge-Base Q&A, a RAG Decision Map, and an Interview Deep Dive

You are handed a knowledge base of five million documents that must answer in about a second, with accuracy as the top priority. How would you design it?给你一个五百万文档、要求秒级响应、准确率优先的知识库场景，你会怎么设计这套系统？
Common in ChinaCommon overseasDeep dive#system-design#scaling#latency-budget
How to reason about it · think before answering
The real subject here is not which technologies you know, it is whether you have a repeatable way to derive a configuration from constraints. Opening with an architecture diagram reads as a memorized answer; the way to score is to turn each constraint into a number first, then let every choice be forced by one of those numbers.
Quantify the three constraints. Five million documents at roughly four or five chunks each is over twenty million chunks; at 1536 float dimensions that is hundreds of gigabytes, so the index does not fit in one machine's memory — that alone settles storage. A one-second budget to first token, with generation typically eating seven or eight hundred milliseconds, leaves only two or three hundred for retrieval. Accuracy first means you may trade latency and money for metrics, but only within that remaining budget.
Now derive each knob from one of those numbers: a dedicated vector store or partitioning, plus half precision (its recall loss usually sits inside run-to-run noise while the index shrinks by about forty percent — essentially free); keep both keyword and vector routes with reciprocal rank fusion, because exact matches on document ids, error codes and names are a permanent blind spot for embeddings; rerank only the top twenty after fusion, since it buys ranking quality at the cost of one synchronous round trip, and a one-second budget affords exactly one.
Then state two things you deliberately do not build, which is the part that reads as field experience. Agentic retrieval is not the default path: its gains concentrate on multi-hop questions while its cost is spread over every question, and it blows a one-second budget outright — the right move is a cheap classifier that routes only the multi-hop minority into the loop. Contextual chunk headers and similar tricks also wait, because they dilute the keyword route while helping the vector route; the directions are opposite, so measure on your own embeddings before committing.
Accuracy first has to become something you can sign off on. That means a golden set of at least a hundred questions with multi-hop and unanswerable each above ten percent, recall and ranking quality read separately, abstention rate on unanswerable questions as its own column, and citations verified by code rather than trusted from the model. Reporting the ugliest column alongside the headline number is far more credible than reporting a single score.
Expected follow-up: how do you build the first index over five million documents? It is a one-off large expense, so batch it, make it resumable, and put content-hash incremental sync in from day one, or every config change means buying the whole corpus again. Push further and you get to rollout: dual-write the new embeddings into a second column, evaluate both columns on the same golden set, then shift traffic, so rollback is a config flip rather than an eight-hour rebuild.
分析过程 · 先想清楚再作答
这题的题眼不在「你会用什么技术」，而在「你有没有一套从约束推配置的方法」。开口就报架构图和技术栈的答案会被判成背方案；拿到分的答法是先把约束翻译成数字，再让每个选择被某个数字逼出来。
先把三个约束量化：五百万文档按一篇四五块估，是两千多万块，单精度 1536 维就是上百 GB，**索引塞不进单机内存**，这一条直接决定了存储选型；秒级响应意味着从收到问题到第一个字的预算大约一秒，而生成本身通常就吃掉七八百毫秒，检索侧只剩两三百毫秒；准确率优先意味着可以拿延迟和钱换指标，但只能换到那两三百毫秒为止。
然后逐项落地，每一项都挂在上面某个数字上：存储上专用向量库或分区加半精度量化（半精度的召回损失通常落在重跑噪声里，索引却小四成，这是白捡的）；检索保留关键词与向量两路加倒数排名融合，因为精确匹配的文档号、错误码、人名是向量的固定盲区；重排只作用于融合后的前二十条——它买的是排序质量，一次同步往返，秒级预算里放得下一次，放不下两次。
接着讲两个「不上」的决定，这一段比上面更能显出做过工程：**Agentic 检索不作为默认路径**，它的收益集中在多跳题上而代价摊给全部问题，秒级预算下更是直接超支——正确做法是先用一次便宜的分类把多跳分流出来，只让那一小部分进循环；**上下文块头之类的手法先不上**，因为它对关键词一路是稀释、对向量一路才是补位，方向相反，得在自己的真实 embedding 上测过再说。
准确率优先必须落成可验收的东西，否则是空话：一份不少于一百题的标准答案集（其中多跳与无答案各占一成以上）、召回率与排序质量分开看、无答案题的拒答率单独一栏、引用由代码回查而不是靠提示词自觉。**报数字时把最难看的那一栏也报出来**，比只报总分可信得多。
可预期的追问：五百万文档怎么建第一版索引？答案是这笔钱是一次性大额支出，要按批做、可断点续跑，并且从第一天就上基于内容指纹的增量同步——否则每次改配置都等于把整个知识库重买一遍。再追问就谈灰度：新旧两套向量双写在两列上，用同一份标准答案集在两列上各跑一遍再切流量，回滚只是改一个配置项。
Key points
Translate constraints into numbers first: twenty million chunks means the index will not fit one machine, and a one-second budget leaves retrieval two to three hundred milliseconds.
Dedicated store or partitions plus half precision; keep keyword and vector routes with RRF, and rerank only the top twenty after fusion.
Name the two things you will not ship: agentic only for a routed multi-hop minority, and chunk headers only after measuring on your own embeddings.
Turn accuracy-first into a hundred-plus question golden set, abstention rate as its own column, and code-verified citations.
First index build is a one-off large expense: batch it, make it resumable, add incremental sync on day one, and dual-write columns for model swaps.
答题要点
先把约束翻译成数字：两千多万块决定索引塞不进单机内存，一秒预算里检索侧只剩两三百毫秒。
存储用专用库或分区加半精度；检索保留关键词与向量两路加倒数排名融合，重排只作用于前二十条。
明确说出「不上」的两项：Agentic 只对分流出来的多跳开，块头这类方向相反的手法先测再说。
准确率优先要落成一百题以上的标准答案集、拒答率单独一栏、引用由代码回查。
第一版建索引是一次性大额支出：分批可续跑，并从第一天就上增量同步；换模型走双写切列。
Looking back at the RAG project you built, which decision would you change now, and why?你做过的这个 RAG 项目里，哪个决定你现在会改？为什么？
Common in ChinaCommon overseasDeep dive#retrospective#evidence#chunking
How to reason about it · think before answering
This looks like a soft question but it separates people sharply. Saying 'nothing yet' admits you never ran a retrospective; a long list of self-criticism reads as poor judgement. What the interviewer is listening for is whether you can chain four things together: the decision, the evidence you had then, the evidence you got later, and your current call.
How to pick: choose a decision that was justified at the time and later overturned by data, not one you always knew was a shortcut. The first proves you measure; the second only proves you were behind schedule. So the answer has a fixed four-part shape — what you chose, on what basis, what you measured later, and what you now believe.
This course supplies a ready example. One day measured that prepending a heading-path header to every chunk left hit rate unchanged, grew index tokens by about ten percent, and pushed the answer document's mean rank from 2.88 to 3.25 — hence 'headers hurt keyword retrieval'. A later day re-ran the same comparison under structure-aware chunking and the rank regression did not reproduce. The reason was the chunker: with fixed-length cuts, chunk boundaries do not line up with section boundaries, so the header injects heading terms that do not belong to that chunk; with structure-aware cuts, each chunk already sits inside one section and the header largely restates what is already there. The correct statement is therefore not 'headers hurt' but 'headers hurt when chunk boundaries are misaligned with document structure'.
The value of the chain is that it demonstrates a reusable habit: attach the premises to every conclusion. Change a premise and you owe a re-run; you may not pair new settings with an old conclusion. The same reasoning yields a second example: an earlier claim that 'the vector route is clearly a net gain' collapsed once the two routes were counted separately before fusion — the vector-only candidates were a small share and contained the answer document zero times, so the improvement was never semantic at all.
Expected follow-up: how will you avoid this class of error in future? Give two concrete practices. Write the bound premises next to every number — corpus, question set, budget, chunker. And before publishing any conclusion, ask whether it was measured by this experiment or forced by the structure of the implementation; the first needs its boundaries stated, only the second can be asserted flatly.
分析过程 · 先想清楚再作答
这题看着是软性问题，其实区分度极高。答「暂时没有」等于承认没做过复盘；答成一长串自我批评又会显得没有判断力。面试官真正在听的是：你能不能把一个决定、它当时的依据、后来的证据、以及新的判断，四样东西串成一条链子说清楚。
怎么拆：挑一个**当时有理由、后来被数据推翻**的决定，而不是一个「当时就知道是凑合」的决定。前者证明你有量化的习惯，后者只证明你赶过工期。所以答案的骨架固定是四段——当时选了什么、依据是什么、后来量到了什么、现在的判断是什么。
本课里有一个现成的样本：某一天先量到「给每个块拼上标题块头之后，命中率不变、索引 token 涨一成、答案文档平均名次从 2.88 退到 3.25」，据此写下「块头对关键词检索是负收益」。后来换成按文档结构切块再复核，这条名次退化**没有复现**。原因是切法变了：固定长度硬切时块边界跟小节边界不对齐，块头会把不属于这一块的标题词塞进来；按结构切时块本身就落在一个小节里，块头补的信息跟块里已有的高度重合。所以正确的表述不是「块头有害」，而是「**块头在块边界与结构不对齐时才有害**」。
这条链子的价值在于它演示了一个可复用的动作：**给每个结论标出它绑定的前提**。前提变了就要重跑，不能拿新配置去配旧结论。顺着这个思路还能给出第二个例子：曾经写过「向量侧确实是正收益」，后来把融合前的两路拆开数了一遍才发现，向量路独有的候选只占很小一部分，其中含答案文档的次数是零——那个「涨」根本不是语义检索带来的，于是这条结论被自己推翻。
可预期的追问：那你以后怎么避免这类错误？答两条具体的：一是每个数字旁边写清它绑定了哪几个前提（语料、题集、预算、切法），二是报结论前先问自己一句「这是这次实验测出来的，还是这个结构必然导致的」——前者要标边界，后者才能直接讲。
Key points
Structure the answer in four beats: the choice, the evidence then, the evidence later, the call now.
Pick a decision that was defensible at the time and later overturned by data, not one you knew was a shortcut.
Worked example: 'headers hurt keyword retrieval' was corrected to 'headers hurt when chunk boundaries misalign with structure', because the chunker premise changed.
Record the premises bound to every number; when a premise changes you owe a re-run rather than a reinterpretation.
Classify before asserting: measured by this experiment, or forced by the implementation's structure — the former needs its boundaries stated.
答题要点
答案要串成四段：当时选了什么、依据是什么、后来量到了什么、现在的判断是什么。
挑一个当时有理由、后来被数据推翻的决定，而不是一个当时就知道在凑合的决定。
样本：块头从「对关键词检索有害」修正成「块边界与结构不对齐时才有害」，因为切法这个前提变了。
每个数字旁边写清它绑定的前提；前提变了就必须重跑，不能新配置配旧结论。
报结论前先分类：这是实验测出来的，还是实现结构必然导致的——前者要标边界。
How do you convince a non-technical stakeholder that your retrieval system actually got better?怎么向不懂技术的业务方证明你的检索系统真的变好了？
Common in ChinaCommon overseasBasic#evaluation#stakeholder-communication#abstention
How to reason about it · think before answering
This is a communication question whose scoring hinges on technical judgement: which numbers you choose to show reveals whether you understand the metrics yourself. Dumping recall, nDCG and MRR on a business stakeholder reads as tone-deaf; saying 'user feedback improved' reads as unmeasured.
Start from a principle: show them something they can adjudicate themselves. They cannot judge normalized discounted cumulative gain, but they can absolutely judge 'out of these hundred real questions, how many did it answer correctly, how many wrongly, and how many did it honestly decline'. So the external framing is three numbers — correct, wrong, declined — and they sum to one hundred.
The crucial move is separating wrong from declined, and it is the fastest way to earn trust: saying 'not found' is a correct output, not a failure; the failure is inventing an answer when nothing was found. Teams that report a single 'accuracy' number can be gamed by a system that learns to decline everything, which is why all three must appear side by side.
Then supply checkable evidence rather than only numbers: take ten real questions and show before-and-after answers with clickable citations on every claim. A stakeholder who opens the source and verifies one claim is more convinced than by any percentage, and the exercise doubles as the human spot-check you need anyway to calibrate whether your model judge is trustworthy.
There is a lesson from this course worth volunteering: a column of perfect scores means the ruler is broken. Our questions were written backwards from the corpus, lexical overlap is unusually high, and mean reciprocal rank sits at exactly 1.0000. Showing that to a stakeholder only invites the misreading that you are already perfect, when in fact the metric has saturated. When a metric hits the ceiling, the response is to make the questions harder.
Expected follow-up: how do you get the business side involved? One very practical answer: let them supply questions. Every production miss gets appended to the golden set, so the evaluation set grows rather than being built once. Then each release can point at 'the question you raised last month now answers correctly', which lands better than any status report.
分析过程 · 先想清楚再作答
这题在考沟通，但拿分点在技术判断上：你选哪几个数字给业务方看，暴露了你自己有没有看懂这些指标。把召回率、nDCG、MRR 一股脑摊出去的答法会被判成不懂受众；只说「用户反馈变好了」又会被判成没有度量。
先立一条原则：**给业务方看的必须是他们能自己判断对错的东西**。归一化折损累计增益他们没法判断，而「这一百个真实问题里，系统答对了多少、答错了多少、老老实实说查不到了多少」他们一眼就能判断。所以对外的口径应该是三个数：答对率、答错率、拒答率，而且三个加起来是一百。
关键是把**答错和拒答分开**。这一条最能建立信任：查不到就说查不到不是故障，是正确输出；真正的故障是查不到还编一段。很多团队只报「准确率」，结果一个学会了一直拒答的系统能刷出满分——所以这三个数必须并排出现，缺一个都能被骗。
然后给可核对的证据，而不是只给数字：**挑十条真实问题做前后对照**，各贴出改动前和改动后的回答，每句结论后面挂着可点开的引用。业务方点开原文核对一遍，比看任何百分比都有说服力，而且这个动作顺带完成了一次人工抽检——你自己也需要它来校准模型裁判靠不靠谱。
本课里有一条要主动说的教训：**一列全是满分说明尺子坏了**。我们的题目是从语料反向出的，字面重合度过高，平均倒数排名恒为 1.0000。这个数字拿给业务方看，只会换来一次「那你们已经完美了」的误会，而它其实是指标饱和。指标撞天花板时该做的是把题目出难一点。
可预期的追问：那怎么让业务方参与进来？答一条很实用的：让他们提供题目。把线上答错的问题一条条补进标准答案集，评估集是长出来的，而不是一次性造好的；这样每一次改进都能指着「你上次提的那个问题现在答对了」，比任何汇报都直接。
Key points
Externally report three numbers they can adjudicate: correct, wrong, declined — summing to one hundred.
Keep wrong and declined separate; a single accuracy number is gamed by a system that learns to decline everything.
Pair it with ten before-and-after real questions, every claim carrying a citation they can open and verify.
Volunteer the saturation caveat: a column of perfect scores means a broken ruler, and the fix is harder questions.
Let stakeholders contribute questions; append every production miss to the golden set so it grows over time.
答题要点
对外只用三个他们能自己判断的数：答对率、答错率、拒答率，三者相加为一百。
答错和拒答必须分开——查不到就说查不到是正确输出，只报一个准确率会被「一直拒答」刷满分。
配十条真实问题的前后对照，每句结论挂可点开的引用，让他们自己核对原文。
主动说明指标饱和：某一列恒为满分是尺子坏了，不是系统完美，该做的是把题目出难一点。
让业务方提供题目，把线上答错的问题补进标准答案集——评估集是长出来的。
Users report that your live RAG system 'answers inaccurately'. What is your triage order?RAG 系统上线后用户反馈「答得不准」，你的排查顺序是什么？
Common in ChinaCommon overseasIntermediate#debugging#failure-modes#observability
How to reason about it · think before answering
This one is almost guaranteed to be asked, and most people answer with a flat list of possibilities: maybe chunking, maybe the prompt, maybe the model. A list is not triage. Triage means an order, a decision rule at each step, and each step eliminating half the search space.
First decompose the complaint. 'Inaccurate' hides at least four distinct failures whose fixes do not transfer: off-topic answers, partial answers, misaligned citations, and stale content. So the first action is not to change a setting, it is to obtain the specific question and answer and classify it into one of those four.
Then give the order along with its justification: read the pipeline right to left, fix it left to right. Right to left because the generated answer is what you see first; left to right because upstream errors are amplified downstream — no prompt can recover a document retrieval never fetched. Concretely: dump the candidate pool and the final context for that question, and check whether the answer document is in the pool at all. Absent means a retrieval debt; present but below the admission gate means a gate debt; admitted but never packed into the context budget means chunks too large or budget too small; all present and still unused means it is finally a generation problem.
One detail worth volunteering because it is easy to get wrong: for multi-hop questions, diagnose the documents that are missing, not whether any one of them was retrieved. In our experiment one question needed two documents; the first ranked first every time and the second never entered the candidate pool at all. Judging by 'any of them' labels it a budget problem, and you can spend a full day tuning budgets to no effect. This distinction only occurs to someone who has actually triaged question by question.
The fourth class, stale content, happens outside the question path and has its own rule: first check whether reconciliation even noticed the edit (was the content hash computed after line-ending normalization?), then check whether the cache key includes the index version and the permission scope. 'When must this expire' is equivalent to 'is that thing part of the key' — leave something out of the key and changes to it will never invalidate the entry.
Expected follow-up: how do you stop relying on manual triage? Build the classification into the evaluation panel so every missed question is automatically labelled with one of the four classes, and report it per tenant. A global average dilutes one customer's collapse across the whole population, and that customer is exactly the one who will file the complaint.
分析过程 · 先想清楚再作答
这题几乎是必考题，而绝大多数人答成一堆并列的可能性：可能是切块问题、可能是提示词问题、可能是模型不行。并列不是排查，排查的意思是**有顺序、有判据、每一步能把可能性砍掉一半**。
先把「答得不准」这四个字拆开——它至少塞了四种病，而且修法互不通用：答非所问、只答得出片段、引用错位、更新不生效。所以第一个动作不是改配置，是**拿到具体的问题和回答，把它归到这四类里的一类**。
然后给顺序，而且要说清顺序的理由：**排查从右往左看、修复从左往右修**。从右往左是因为你最先看到的是生成结果；从左往右是因为上游的错会被下游放大——检索没捞到的东西，再好的提示词也救不回来。具体走法是：打印这一问的候选池和最终上下文，先看答案文档在不在候选池里。不在，是检索的债；在候选池但没过准入门槛，是门槛的债；过了门槛却没装进上下文预算，是块太大或预算太小；都进了而模型没用上，才轮到生成侧。
这里有一个容易写错的细节值得主动讲：**多跳题的诊断对象是缺的那几篇，不是「有没有捞到任意一篇」**。我们实验里有一道题要同时命中两篇，第一篇稳稳排第一、第二篇一次都没进候选池；用「任意一篇」去判会把它归成预算问题，然后你去调预算，调一整天也没用。这一条区分度很高，因为它只有真的按题排查过才想得到。
第四类「更新不生效」发生在问答之外，判据是另一条：先看对账认没认出这篇改了（内容指纹算之前有没有做换行归一化），再看缓存的 key 里有没有把索引版本和权限范围算进去。「什么时候必须失效」等价于「key 里有没有把那样东西算进去」，key 少放一样，那样东西变了缓存就不会失效。
可预期的追问：怎么让这套排查不靠人肉？答案是把分类做进评估面板——每一道没中的题自动标出它属于四类中的哪一类，并按租户分开统计。全局平均会把单个客户的塌方按人头摊薄，而线上会投诉的恰恰是那个客户。
Key points
Classify the complaint into four failures first — off-topic, partial, misaligned citation, stale — because their fixes do not transfer.
Read right to left, fix left to right: dump the candidate pool and final context and find which layer the answer document stalls at.
The four rules in order: never retrieved, retrieved but below the gate, admitted but squeezed out of the budget, packed but unused by the model.
For multi-hop, diagnose only the missing documents; judging by 'any one retrieved' mislabels a never-retrieved case as a budget problem.
For stale content, check reconciliation and the cache key: what must expire is exactly what the key must contain.
答题要点
先把「答得不准」归类成四种病：答非所问、只答得出片段、引用错位、更新不生效——修法互不通用。
排查从右往左看、修复从左往右修：先打印候选池与最终上下文，看答案文档卡在哪一层。
四层判据依次是：没进候选池、进了没过门槛、过了没装进预算、都进了模型没用上。
多跳题只诊断缺的那几篇；用「有没有捞到任意一篇」会把「根本没捞到」误判成预算问题。
「更新不生效」查对账与缓存 key：什么时候必须失效，等价于 key 里有没有算进那样东西。
If you could only fund three changes to improve an existing RAG system, which three would you pick and why those three?如果预算只够做三件事来提升一个已有 RAG 系统的效果，你选哪三件？为什么是这三件？
Common in ChinaCommon overseasIntermediate#prioritization#evaluation#abstention
How to reason about it · think before answering
This tests prioritisation, not breadth. Answering with a list of techniques — add reranking, add hybrid retrieval, add query rewriting — almost always loses points, because it skips a prerequisite: how do you know those three help your system? That is precisely the sentence the interviewer is waiting for.
So the first item has to be building evaluation, with a reason specific enough to be unarguable: without a scale, you cannot tell whether the other two helped or hurt; with one, every subsequent spend has a measurable return. It is also cheap — the three retrieval metrics are pure local computation, run in seconds, cost nothing, and can gate every commit; the only real effort is labelling answer documents once. Include the composition rule: multi-hop and unanswerable each above ten percent, because without the unanswerable class a system that only ever guesses scores perfectly on your report.
Second, move abstention out of the prompt and into code — usually the best return per unit of effort, and the item most often skipped. Writing 'say you don't know' ten times in a prompt buys almost nothing. Citation numbers are a closed set, so checking existence is one line, and adding a substantive-overlap check catches the harder forgery where the number is real but the content is not. Our baseline abstention rate was 0.0 percent: four questions with no answer in the corpus, zero of them declined — a defect that is completely invisible on a report that only shows recall.
Third, look at the failure cases before deciding, which is the actual answer to this question. After reading the panel you land on one of a few branches: a high share of multi-hop means bridging retrieval or a different index structure; queries that miss when phrased differently mean you need the vector route or hybrid retrieval; answers retrieved but never packed into context means reranking or budget. Failure cases first, technique second — we tried five advanced index structures and not one beat the baseline, because our system simply did not have the weakness they address.
Why not the flashier options: agentic retrieval concentrates its gains on multi-hop while spreading cost across every question, and in our measurements turning on every query-side technique produced exactly the same recall as the default configuration while using 2.5 times the model calls and 4.3 times the retrievals. Stacking techniques is easy; explaining why you switched several off is the skill.
Expected follow-up: once the three are done, how do you prove the money was well spent? Toggle each one individually and report three ledgers — how much the metric moved, how much latency moved, how much cost moved. A proposal that reports only the first should not be approved, including your own.
分析过程 · 先想清楚再作答
这题在考优先级判断，而不是知识面。答成「上重排、上混合检索、上查询改写」这类手法清单几乎必然掉分——因为它跳过了一个前提：**你凭什么知道这三件对你的系统有用？** 面试官等的就是这句话。
所以第一件必须是**建评估**，而且理由要具体到不可反驳：没有秤，剩下两件做完你也说不清是变好还是变坏；有了秤，后面每一笔钱都能算回报。而且它便宜——检索侧三个指标是纯本地计算、几秒钟、零成本，能挂进每次提交；花时间的只是给题目标答案文档那一次。顺带说清评估集的配比：多跳与无答案各占一成以上，缺了无答案那一类，一个只会硬答的系统在报表上就是满分。
第二件是**把拒答从提示词搬进代码**，这一件的性价比通常最高而最容易被跳过。提示词里写十遍「找不到就说找不到」增益接近于零；而引用编号是一个闭集，判它存不存在只要一行代码，再加一道「这句话与被引块的实质重合度」就能拦住「编号是真的、内容是假的」那一类。我们实验里的基线拒答率是 0.0%——四道语料里根本没有答案的题一道都没闭嘴，这类缺陷在只报召回率的报表上完全不可见。
第三件要**先看失败案例再决定**，这才是这道题真正的答案。看完面板你会落到其中之一：多跳题占比高就补桥接检索或改索引结构；换个说法就捞不到，说明该上向量那一路或混合检索；答案捞到了却排不进上下文，那是重排或者预算的活。**先有失败案例，再有手法**——我们试过五种高级索引结构，没有一种跑赢基线，因为我们的系统压根没有那些结构要补的短板。
为什么不选那些看起来更亮的：Agentic 检索的收益集中在多跳题上而代价摊给全部问题；「全开」所有查询侧手法在我们的实测里召回率和默认配置一模一样，模型调用却是 2.5 倍、检索次数 4.3 倍。**堆手法很容易，说清楚为什么关掉某几项才是本事。**
可预期的追问：三件做完怎么证明钱花对了？答：每一项单独开关各跑一遍，报三笔账——指标涨了多少、延迟涨了多少、钱涨了多少。只报第一笔的提案不该被批准，包括你自己的。
Key points
First, build evaluation: without a scale the other two changes are unverifiable, and the retrieval metrics are cheap enough to gate every commit.
The golden set must include unanswerable questions, or a system that only ever guesses scores perfectly on your report.
Second, move abstention from the prompt into code: citation numbers are a closed set, and a substantive-overlap check catches real-number-fake-content forgeries.
Third is chosen by the failure cases, not by a list of techniques — failure cases first, index structure or retrieval trick second.
Toggle each change individually and report three ledgers: metric, latency, cost. A proposal reporting only the first should not be approved.
答题要点
第一件是建评估：没有秤，另外两件做完也说不清变好还是变坏；检索侧指标零成本可挂进每次提交。
评估集必须含无答案那一类，否则一个只会硬答的系统在报表上就是满分。
第二件是把拒答从提示词搬进代码：编号是闭集，再加实质重合度就能拦住「编号真、内容假」。
第三件由失败案例决定，不由手法清单决定——先有失败案例，再有索引结构或检索手法。
每一项单独开关跑一遍并报三笔账：指标、延迟、钱。只报第一笔的提案不该被批准。