Interview Bank

Build an AI Short-Drama Production Pipeline With Agents in 14 Days

D5 Voiceover, Subtitles, and Audio Tracks: Multi-Character Voices, Timeline Alignment, and Subtitle Files

• When the synthesized speech and the shot duration disagree, which side do you adjust, and why?语音时长和画面时长对不上，你会调哪一边？为什么？

  Common in ChinaCommon overseasIntermediate#timeline#tts#pipeline-design

  How to reason about it · think before answering

  1. Answering 'stretch the shot' alone scores nothing; the hinge is 'why'. They want the reasoning for which side yields, and whether you see that this choice fixes the order of the whole pipeline.
  2. State the criterion: which distortion does the audience notice? Clipped or sped-up dialogue is audible immediately; a shot running 0.8 seconds long is not. So the picture yields.
  3. Derive the pipeline order from that: generate video at the planned duration, synthesize speech, write the measured duration back onto the shot, and let the editor pad the picture. Why not synthesize first and generate video to fit? Because video APIs expose discrete duration options — you cannot ask for exactly 6.34 seconds.
  4. Add the engineering detail that cannot be skipped: the timeline must use durations measured from the rendered files, never character-count estimates. Estimation error accumulates line by line, and by the tenth line the subtitles visibly race the picture.
  5. Then the exception, which earns points: if a shot has intrinsic rhythm — a beat cut, a transition, an action match — the picture cannot simply be stretched, and the right fix is a shorter line in the script. That is why stretched shots should be flagged for human review rather than silently rewritten.
  6. Expect the follow-up: can't you just nudge the speaking rate? You can, but it costs you — rate changes affect timbre and delivery, and they change duration again, turning a one-way flow into a loop. Make the lead-in and tail padding adjustable and spend that budget before touching the rate.

  分析过程 · 先想清楚再作答

  1. 这题只答「调画面」拿不到分，题眼在「为什么」——面试官要的是让步理由，以及你有没有意识到这个选择会决定整条流水线的排列顺序。
  2. 先给判断依据：哪一边的失真观众察觉得到。台词被切掉、或者被加速到语气变形，观众立刻听得出来；一镜比原计划长零点八秒，观众感觉不到。所以让步的是画面。
  3. 由这条判断反推流水线顺序：画面先按计划时长生成，语音合成完之后由真实时长回写镜头时长，剪辑台再去补足画面。为什么不倒过来先合成语音再按语音时长生成视频？因为视频接口的时长是有限档位的，你没法要求它精确生成 6.34 秒。
  4. 补一条不能省的工程细节：写进时间轴的必须是从落盘文件量出来的真实时长，不能是字数估算。估算误差是逐句累加的，第一句差两百毫秒，第十句就差两秒，成片上表现为字幕跟画面赛跑。
  5. 再说例外，这是加分项：如果这一镜的画面本身有强节奏（比如卡点、转场、动作衔接），画面就不能被随意拉长，这时候要回头改剧本把台词写短，而不是硬拉画面。所以被顶长的镜头应该被标记出来交给人复核，而不是程序默默改掉。
  6. 可以预期的追问：那不能微调语速吗？可以，但语速是有代价的——语速改变会同时改变音质与情绪表现，而且它会反过来再改一次时长，等于把一个单向流程变成了循环。留一点余量的做法是给留白参数一个可调区间，先动留白再动语速。

  Key points

  • Stretch the picture: clipped or sped-up dialogue is instantly audible, while a fraction of a second of extra shot length is not
  • That fixes the pipeline order: generate video at planned duration, synthesize speech, write measured duration back, pad in the edit
  • You cannot invert it and generate video to match speech, because video APIs only expose discrete durations
  • The timeline must use durations measured from rendered files; character-count estimates accumulate error line by line
  • Shots with intrinsic rhythm are the exception, so flag stretched shots for human review instead of silently rewriting them

  答题要点

  • 调画面：台词被切或被加速观众立刻察觉，镜头长零点几秒观众感觉不到
  • 由此定下流水线顺序：画面按计划时长生成，语音合成后回写真实时长，剪辑台补足画面
  • 不能倒过来按语音时长生成视频，因为视频接口的时长只有有限档位
  • 时间轴必须用落盘文件量出的真实时长，字数估算的误差会逐句累加
  • 画面有强节奏的镜头是例外，这类冲突应标记出来交人复核而不是程序默默改掉
• Where do you get subtitle timestamps from, and what do you do when the API does not provide them?字幕的时间戳你会怎么拿？接口不给时间戳时有什么替代方案？

  Common in ChinaCommon overseasIntermediate#subtitles#timeline

  How to reason about it · think before answering

  1. This tests whether you would take a dependency on an optional vendor field. Name both paths and their costs; giving only one invites a follow-up you will not enjoy.
  2. Path one is the API: TTS endpoints often expose a subtitle flag returning sentence- or word-level timestamps. Three problems — it costs an extra request to fetch, the timestamps are relative to that single audio segment, and the field structure varies by vendor. The third is the worst, because it welds your subtitle module to one provider.
  3. Path two is local alignment: you already hold every clip's measured duration and every shot's start time, so accumulating them gives the episode timeline. Zero extra requests, zero vendor coupling, and you control segmentation — one line of dialogue per cue, which is exactly the rhythm short drama wants.
  4. The key insight is that path one does not free you from path two: API timestamps are segment-relative, so you still add the shot's offset within the episode. Since you must write the alignment code anyway, make it the single source of truth.
  5. The implementation has one rule: the subtitle cursor and the shot cursor share one origin and advance together. Add a check that every cue falls inside its own shot — overflow raises no error, it just floats the previous shot's line over the next shot's picture.
  6. Expect the follow-up: what about karaoke-style word-level subtitles? That genuinely requires word-level timestamps from the API. Treat it as an optional enhancement over a local-alignment main path, degrading to sentence level when word data is unavailable.

  分析过程 · 先想清楚再作答

  1. 这题在考你会不会为一个可有可无的厂商字段引入依赖。两条路都要说得出来，还要说清各自的代价，只答一条会被追问到底。
  2. 第一条是接口给：语音合成接口通常有一个字幕开关，返回按句或按词的时间戳。它的问题有三个——要多发一次请求去取内容、时间戳是相对单段音频的、字段结构随厂商变化。第三条最要命，因为它让你的字幕模块和某一家厂商绑死了。
  3. 第二条是本地对齐：你手里已经有每段音频的真实时长和每一镜的起始时刻，累加就是整集时间轴。它零额外请求、零厂商依赖，而且断句由你自己控制——按台词行断，一句一条，天然符合短剧节奏。
  4. 关键在于**就算用第一条也逃不掉第二条**：接口给的是段内相对时间，你仍然要加上这一镜在整集里的偏移。所以本地对齐这套代码无论如何都要写，那不如让它成为唯一的真相来源。
  5. 对齐的实现只有一个要点：字幕游标和镜头游标必须共用同一个原点，逐镜推进。再配一个自检——每条字幕必须落在它所属的那一镜内，越界不会报错，只会让上一镜的台词飘到下一镜的画面上。
  6. 可以预期的追问：那按词级时间戳做卡拉OK式字幕呢？那种效果确实必须依赖接口的词级时间戳，本地对齐做不了。这时的正确做法是把它做成一个可选增强，主链路仍然走本地对齐，拿不到词级数据就降级成句级。

  Key points

  • Two sources: timestamps returned by the API, and local alignment accumulated from measured audio durations
  • The API path costs an extra request, gives segment-relative timestamps, and couples you to one vendor's field structure
  • Local alignment needs no extra request and no vendor coupling, and lets you segment per line of dialogue
  • Even with API timestamps you must add each shot's offset within the episode, so the alignment code is unavoidable anyway
  • The implementation rule is one shared origin for the subtitle and shot cursors, plus a check that each cue stays inside its own shot

  答题要点

  • 两条来源：接口返回的时间戳，以及由音频真实时长本地累加对齐
  • 接口那条的代价是多一次请求、时间戳只相对单段音频、字段结构跟厂商绑定
  • 本地对齐零额外请求零厂商依赖，断句按台词行控制，符合短剧节奏
  • 即使用接口时间戳也仍要自己加上这一镜在整集里的偏移，所以本地对齐代码无论如何都得写
  • 实现要点是字幕游标与镜头游标共用同一原点，并自检每条字幕是否落在它所属的镜头内

D6 The Editing Bay: Assembling Footage Into One Vertical Cut With ffmpeg

• An auto-generated episode comes out with audio and video out of sync. What is your debugging order, and why that order?一集自动生成的短剧成片出现音画不同步，你的排查顺序是什么？为什么是这个顺序？

  Common in ChinaCommon overseasIntermediate#debugging#av-sync#timeline

  How to reason about it · think before answering

  1. The question is about ordering, not about listing causes. The interviewer wants to see you rank checks by hit rate divided by cost, not enumerate everything you can think of.
  2. Ask yourself first: where does time come from in this pipeline? If the answer is 'a structured timeline table', then step one is comparing planned durations in that table against the real durations of the media files. Highest hit rate, lowest cost, one ffprobe call.
  3. Step two is the upstream artifacts: when the voice track is longer than the shot, the line gets cut off. It sounds almost identical to drift but the root cause is different, and it should have been caught with a warning when the timeline was built.
  4. Step three is the compose stage: stream-copy concatenation requires identical parameters across segments, and misaligned timestamps shift things; adding crossfades shortens the final cut, so subtitles drift progressively unless their timecodes are recomputed.
  5. Also mention a general move: when all three fail, stop staring at the final cut and play the normalized per-shot segments to narrow the problem to one shot. Always shrink the search space before guessing.
  6. Expect the follow-up 'how do you stop relying on human ears'. Answer: assert at timeline-build time when planned and actual durations diverge beyond a threshold, and automatically verify that the final cut's duration matches the timeline total.

  分析过程 · 先想清楚再作答

  1. 这题的题眼在「顺序」两个字，不在「有哪些原因」。面试官想看的是你会不会按「命中率乘以排查成本」来排，而不是把想到的原因罗列一遍。
  2. 先问自己一个问题：这条流水线上，时间是从哪里来的？如果答案是「一张结构化的时间轴表」，那么第一步必然是拿表里的计划时长和素材文件的真实时长去对——这一步命中率最高、成本最低，一条 ffprobe 就能查完。
  3. 第二步查上游的产物本身：配音时长超过镜头时长时，台词会被截断，听感和不同步几乎一样，但根因完全不同。这类冲突应该在生成时间轴时就打警告，而不是留到成片阶段靠耳朵发现。
  4. 第三步才查合成环节：流拷贝拼接要求各段参数一致，时间戳对不齐就会错位；加了转场则成片整体变短，字幕若没跟着重算，表现为越到后面偏得越多。
  5. 还有一条通用招式值得说出来：三步都查不出来时，不要在成片里死磕，去播归一化之后的单镜片段，把问题缩小到某一镜身上。排查多段合成的问题永远优先缩小范围。
  6. 可预期的追问是「怎么让这类问题不再靠人耳发现」。答：在时间轴生成阶段加断言（计划时长与素材真实时长的偏差超过阈值就失败），并把成片时长与时间轴总时长的一致性做成自动校验。

  Key points

  • Start with the timeline table: compare planned durations against the media files' real durations. Highest hit rate, cheapest check.
  • Then check whether the voice track exceeds the shot duration and truncates the line. That should be warned about at timeline-build time.
  • Only then look at compose: concat method, timestamp alignment, and crossfades shortening the cut without recomputed subtitle timecodes.
  • General move: play the per-shot normalized segments to isolate one shot instead of guessing on the final cut.
  • Long term, turn duration consistency into assertions and automated checks rather than relying on ears.

  答题要点

  • 先查时间轴表里的计划时长与素材真实时长是否一致，这一步命中率最高、成本最低。
  • 再查配音是否超出镜头时长导致台词被截断，这类问题应在生成时间轴时就报警告。
  • 最后查合成环节：拼接方式、时间戳对齐、转场是否让成片变短而字幕没重算。
  • 三步之外的通用招式：播单镜片段把问题缩小到某一镜，不要盯着最终产物猜。
  • 长期方案是把时长一致性做成断言与自动校验，不靠人耳兜底。