Dayward AI

Interview Bank

328 questions total; 2 shown with current filters.

Tag
361 more tags
#api-design9#coding-agent9#distributed-systems8#multi-agent8#rag8#chunking7#debugging7#pipeline-design7#structured-output7#agent-loop6#mcp6#operations6#prompt-injection6#sse6#tool-calling6#tool-design6#context5#context-engineering5#deployment5#embeddings5#hybrid-search5#message-bus5#scheduling5#system-prompt5#agentic-rag4#behavioral4#client4#concurrency4#consistency4#framework-design4#ingestion4#prompt-engineering4#rate-limiting4#retrieval4#routing4#trade-offs4#abstention3#agents-sdk3#caching3#communication3#context-window3#cost-control3#data-quality3#failure-modes3#image-generation3#interview-prep3#langgraph3#latency3#llm-as-judge3#long-context3#model-migration3#model-routing3#orchestration3#ordering3#progressive-disclosure3#prompt-basics3#prompt-caching3#protocol3#provider-abstraction3#ranking3#recall3#redis-streams3#responses-api3#resume3#retry3#runtime3#scalability3#skills3#state-management3#statelessness3#subagents3#timeline3#versioning3#workflow-engine3#access-control2#agent-design2#async-task2#auth2#checkpointing2#citation-verification2#citations2#claude-md2#code-review2#compaction2#context-rot2#contextual-retrieval2#cost-tradeoff2#data-modeling2#database2#distribution2#fallback2#ffmpeg2#golden-set2#grounding2#interview-process2#llm-basics2#long-term-memory2#media-pipeline2#memory2#multi-hop2#multi-tenancy2#oauth2#prioritization2#project-storytelling2#prompt-techniques2#query-rewriting2#react2#refusal2#reproducibility2#rerank2#retrieval-quality2#retrospective2#scripts2#sharding2#state-machine2#tool-permissions2#tooling2#tools2#transport2#tts2#ux2#vector-database2#verification2#workflow2#agent-basics1#agent-quality1#agent-sdk1#agents-md1#altitude1#analytics1#approvals1#architecture-review1#async1#atomicity1#attention-budget1#av-sync1#backoff1#bi-encoder1#bm251#budget-control1#build-vs-buy1#cancellation1#candidate-selection1#capacity-planning1#career1#chain-of-thought1#ci1#circuit-breaker1#claude-code1#cli-design1#client-integration1#cloud1#compliance1#compression1#configuration1#confused-deputy1#consistent-hashing1#content-hash1#content-safety1#context-assembly1#context-compression1#context-management1#copyright1#coreference1#correctness1#cost-accounting1#cost-analysis1#cost-optimization1#cross-encoder1#cross-model1#customer-support1#dag1#decision-making1#decomposition1#degradation1#deliberate-practice1#design1#diagnostics1#dimensions1#docker1#documentation1#embedding-migration1#encoding1#engineering-judgement1#engineering-tradeoffs1#error-propagation1#escalation1#eval1#event-driven1#evidence1#failure-analysis1#fairness1#faithfulness1#fan-out1#feedback-loop1#fencing-token1#few-shot1#filter-pushdown1#filtering1#fine-tuning1#forking1#four-elements1#framework-selection1#frontend1#global-market1#graph-rag1#guardrails1#hallucination1#handoff1#handoffs1#headless1#hnsw1#hooks1#human-in-the-loop1#hybrid1#hyde1#incremental-recompute1#incremental-sync1#index-maintenance1#index-routing1#indexing1#information-retrieval1#instruction-hierarchy1#integration1#intent-routing1#interrupt-merge1#invalidation1#isolation1#iterative-scan1#ivfflat1#json-parsing1#json-schema1#just-in-time1#jwt1#knowledge-organization1#labeling1#latency-budget1#lease1#least-privilege1#llm-output-quality1#long-session1#loop-guard1#maintenance1#mcp-basics1#mental-model1#messages-api1#metadata1#methodology1#metrics1#migration1#mobile1#model-selection1#moderation1#modularity1#multi-turn1#multimodal1#nodejs1#normalisation1#notifications1#ocr1#offline-testing1#openai1#overlap1#pagination1#parent-child1#pdf-parsing1#performance1#permissions1#persistence1#pgvector1#pipeline-reliability1#portfolio1#primitives1#priority-queue1#proactive-messaging1#product-engineering1#production-readiness1#prompt1#prompt-assembly1#prompt-bloat1#prompt-design1#prompt-limits1#prompt-surface1#prompt-template1#prompt-versioning1#prompting1#protocol-versions1#quality1#quality-check1#quantization1#query-transformation1#quiet-hours1#rag-basics1#rank-fusion1#reasoning1#reconnect1#redis1#reflection1#replay1#reporting1#retrieval-failure1#retrieval-metrics1#retry-semantics1#retry-strategy1#review1#risk-assessment1#rollback1#rollout1#rrf1#safety1#sandbox1#sandboxing1#scaling1#schema-design1#schema-validation1#scoping1#secrets-management1#self-assessment1#self-introduction1#self-presentation1#self-reflection1#server-design1#service-architecture1#session-management1#sessions1#similarity1#skill-authoring1#skill-description1#skill-design1#spec1#split-brain1#stakeholder-communication1#star1#state-persistence1#stateless1#stdio-transport1#stopping-criteria1#storytelling1#subagent1#subscriptions1#subtitles1#task-graph1#team-governance1#test-set1#test-strategy1#testing1#thresholds1#timezone1#token-accounting1#token-budget1#tool-budget1#tool-execution1#tool-naming1#tool-schema1#trust-boundary1#tuning1#validation1#vector-index1#workflow-design1#xml-tags1#zero-downtime1

From Frontend Engineer to Agent Engineer in 30 Days

D21 Evaluation and Observability: a Golden Set, LLM-as-Judge, Tracing, a Failure-Rate/Cost Dashboard; Pi vs. LangGraph Summary; Week Three Retrospective

  • What does observability look like for a multi-agent system, and how does it differ from a single agent?多 Agent 系统的可观测性要看哪些东西?和单 Agent 有什么不一样?
    Common in ChinaCommon overseasIntermediate#observability#tracing#distributed-systems

    How to reason about it · think before answering

    1. The hinge is differ. Saying add logs and metrics is a non-answer; name the structural difference.
    2. In one sentence: a single agent's call is a line, a multi-agent request is a tree. One request goes supervisor routing, planner splitting into three, three executors in parallel, a critic rejecting one, that one rerunning, then aggregation — flattened by time you cannot see nesting or which two ran concurrently.
    3. So spans must carry a parent pointer; that is the whole game. With it you have a tree, without it a flat list where you know what happened but not what triggered what. A span needs surprisingly few fields — id, parent, name, start and end, a few attributes — to reconstruct the entire tree.
    4. How the parent propagates is itself an interview point: do not thread a parentSpanId parameter through every function, because each new node then changes a signature and one omission breaks the chain. Use the language's implicit context — AsyncLocalStorage in JS, contextvars in Python, TaskLocal in Swift, and ScopedValue or ThreadLocal with explicit propagation across thread pools in Java.
    5. Then the four questions a dashboard must answer: how much is wrong (pass rate, routing accuracy, degradation rate, fallback rate), where is it slow (p50/p95), what did it cost, and which role spent the money (cost attributed per node). That last one is multi-agent specific and the most actionable — measured, executor nodes took over a third of spend, telling you immediately where to optimise.
    6. One foundational point: the dashboard is not a second instrumentation layer, it is an aggregation of traces. The same raw data read across is a tree and stacked up is a dashboard. Two separate sources will eventually disagree, after which nobody trusts either.
    7. Finally, tie back to routing: the routing decision is made by a model and the same sentence may route differently next time, so the routing rationale must be recorded — if you do not capture it then, that judgement is gone forever. It is the easiest thing to omit and the thing most needing post-hoc audit.

    分析过程 · 先想清楚再作答

    1. 题眼在「不一样」。答「加日志加监控」等于没答,要说清结构上的差别。
    2. 结构差别一句话:**单 Agent 的一次调用是一条线,多 Agent 是一棵树。** 一次请求走监督者路由、规划者拆三件、三个执行者并行、评审者打回一件、那件重跑、最后汇总——按时间平铺看不出谁在谁里面,也看不出哪两个是并行的。
    3. 所以 span 必须带**父指针**,这是全部关键:有它才是树,没它只是一张平铺列表,你知道发生过什么,却不知道谁触发了谁。一条 span 的字段少得出奇——id、父指针、名字、起止时刻、几个属性,就够还原整棵树。
    4. 父子关系怎么传下去也是个考点:**不要在每个函数上加一个 parentSpanId 参数**,每加一个节点都要改签名、漏一处断一截。用语言自带的隐式上下文——JS 的 AsyncLocalStorage、Python 的 contextvars、Swift 的 TaskLocal,Java 用 ScopedValue 或 ThreadLocal 配合线程池的显式传播。
    5. 然后说面板要回答哪四个问题:错了多少(通过率、路由准确率、降级率、兜底率)、慢在哪(p50/p95)、花了多少、**钱花在哪个角色身上**(按节点分摊)。最后一样是多 Agent 特有的,也最有用——实测执行者节点占了成本三分之一强,一眼就知道压成本先压哪儿。
    6. 还有一条地基性的:**面板不是另一套埋点,是 trace 的聚合**。同一份原始数据横着看是树、竖着堆是面板。两套数据来源迟早会对不上,然后没有人相信任何一个。
    7. 最后回指路由:路由决策是模型做的,同一句话下次未必给同样的答案,所以必须把**路由理由**一起记下来——当时不记,那次判断就永远丢了。这是多 Agent 里最容易漏、又最需要事后审计的一条。

    Key points

    • Structural difference: a single agent call is a line, multi-agent is a tree (route, split, parallel execute, critic reject, rerun, aggregate)
    • Spans need a parent pointer, or you have a flat list showing neither nesting nor parallelism
    • Propagate parentage through implicit context (AsyncLocalStorage / contextvars / TaskLocal), not a parameter on every signature
    • The dashboard answers four questions: how much is wrong, where it is slow, what it cost, and which role spent it — the last is multi-agent specific and most actionable
    • The dashboard must be an aggregation of traces, not separate instrumentation; two sources will disagree
    • Record the routing rationale: routing is a model decision, and uncaptured it is lost forever

    答题要点

    • 结构差别:单 Agent 一次调用是一条线,多 Agent 是一棵树(路由→拆分→并行执行→评审打回→重跑→汇总)
    • span 必须带父指针,否则只是平铺列表,看不出嵌套关系也看不出并行
    • 父子关系用语言自带的隐式上下文传(AsyncLocalStorage / contextvars / TaskLocal),不要在每个函数签名上加参数
    • 面板回答四个问题:错了多少、慢在哪、花了多少、钱花在哪个角色身上(最后一个是多 Agent 特有且最有用)
    • 面板必须是 trace 的聚合而不是另一套埋点,两套数据源迟早对不上
    • 路由理由必须记下来:路由是模型做的决策,当时不记那次判断就永远丢了

Mastering Codex and the OpenAI Agents SDK in 5 Days

D4 The OpenAI Agents SDK: Agents, Handoffs, Guardrails, Sessions, Tracing

  • Both Agents SDK sessions and the Responses API's previous_response_id remember multi-turn state. How do you choose, and what role does tracing play?Agents SDK 的 session 和 Responses API 的 previous_response_id 都能记住多轮,怎么选?tracing 在这里起什么作用?
    Common in ChinaCommon overseasIntermediate#agents-sdk#sessions#tracing

    How to reason about it · think before answering

    1. This probes your sensitivity to who holds the state, the SDK-level echo of 'you carry the history yourself'.
    2. Ask three questions: can the history be audited, trimmed or replayed, and kept within data-residency rules? previous_response_id keeps history server-side with minimal requests but answers all three poorly; sessions keep it in your store and answer all three, at the cost of managing storage.
    3. Conclude: prototypes and internal tools take previous_response_id; user-facing production keeps its own copy, for which sessions are the ready-made path; both can coexist.
    4. Of the four session operations, pop_item deserves mention: removing the last turn to honor a user's undo is only possible when you own the history.
    5. Tracing makes multi-agent behavior explainable: on by default, one trace per run recording turns, tool calls, handoffs and guardrail results; group a conversation with withTrace or group_id; disable via env var or swap in your own exporter for sensitive data.
    6. Expect the follow-up: does tracing ship user data out? By default it goes to the platform dashboard, so regulated settings must disable it or replace the processors.

    分析过程 · 先想清楚再作答

    1. 这题考的是对「状态放在谁手里」的敏感度,是 30 天课 D1「历史靠你自己搬」在 SDK 层的翻版。
    2. 拆法是问三件事:历史能不能审计、能不能裁剪或重放、能不能满足数据驻留要求。previous_response_id 的历史在服务端,请求最小、代码最简,但三个问题都答不好;session 的历史在你手里(内存、SQLite、Redis),三个都能做,代价是自己管存储。
    3. 结论:原型与内部工具用 previous_response_id 省事;面向用户的生产系统至少自己落一份历史,session 是现成的落法;两者可以同时用。
    4. session 的四个接口(取、追加、弹出最后一条、清空)里 pop_item 值得点出:用户撤回上一句时把最后一轮拿掉再重跑,这是自己持有历史才能做的事。
    5. tracing 的作用是让多 Agent 系统的行为可解释:默认开启,每次 run 一条,记录每轮、每次工具调用、交接与护栏判断;用 withTrace 或 group_id 把一段对话归到一起;敏感数据场景用环境变量关掉或换成自己的导出器。
    6. 可预期的追问:tracing 会不会把用户数据传出去?默认会传到平台面板,所以合规场景要么关、要么 setTraceProcessors 换成自己的后端。

    Key points

    • previous_response_id keeps history server-side, small and simple, but weak on audit, trimming and residency
    • Sessions keep history in your store, auditable and replayable, with pop_item for undo; production keeps its own copy
    • Tracing is on by default, one trace per run, capturing turns, tools, handoffs and guardrails, grouped via group_id
    • For sensitive data disable it with OPENAI_AGENTS_DISABLE_TRACING or swap in your own exporter

    答题要点

    • previous_response_id 历史在服务端,请求小代码简,但难审计、难裁剪、难满足数据驻留
    • session 历史在自己手里,可审计可重放,pop_item 支持撤回;生产至少自己落一份
    • tracing 默认开、每次 run 一条,记录每轮工具、交接与护栏,用 group_id 归组
    • 敏感数据场景用 OPENAI_AGENTS_DISABLE_TRACING 关掉或换成自己的导出器