面试题库

14 天用 Agent 搭一条 AI 短剧生产线

D5 配音、字幕与音轨：多角色语音、时间轴对齐与字幕文件

• 字幕的时间戳你会怎么拿？接口不给时间戳时有什么替代方案？Where do you get subtitle timestamps from, and what do you do when the API does not provide them?

  国内高频海外高频进阶#subtitles#timeline

  分析过程 · 先想清楚再作答

  1. 这题在考你会不会为一个可有可无的厂商字段引入依赖。两条路都要说得出来，还要说清各自的代价，只答一条会被追问到底。
  2. 第一条是接口给：语音合成接口通常有一个字幕开关，返回按句或按词的时间戳。它的问题有三个——要多发一次请求去取内容、时间戳是相对单段音频的、字段结构随厂商变化。第三条最要命，因为它让你的字幕模块和某一家厂商绑死了。
  3. 第二条是本地对齐：你手里已经有每段音频的真实时长和每一镜的起始时刻，累加就是整集时间轴。它零额外请求、零厂商依赖，而且断句由你自己控制——按台词行断，一句一条，天然符合短剧节奏。
  4. 关键在于**就算用第一条也逃不掉第二条**：接口给的是段内相对时间，你仍然要加上这一镜在整集里的偏移。所以本地对齐这套代码无论如何都要写，那不如让它成为唯一的真相来源。
  5. 对齐的实现只有一个要点：字幕游标和镜头游标必须共用同一个原点，逐镜推进。再配一个自检——每条字幕必须落在它所属的那一镜内，越界不会报错，只会让上一镜的台词飘到下一镜的画面上。
  6. 可以预期的追问：那按词级时间戳做卡拉OK式字幕呢？那种效果确实必须依赖接口的词级时间戳，本地对齐做不了。这时的正确做法是把它做成一个可选增强，主链路仍然走本地对齐，拿不到词级数据就降级成句级。

  How to reason about it · think before answering

  1. This tests whether you would take a dependency on an optional vendor field. Name both paths and their costs; giving only one invites a follow-up you will not enjoy.
  2. Path one is the API: TTS endpoints often expose a subtitle flag returning sentence- or word-level timestamps. Three problems — it costs an extra request to fetch, the timestamps are relative to that single audio segment, and the field structure varies by vendor. The third is the worst, because it welds your subtitle module to one provider.
  3. Path two is local alignment: you already hold every clip's measured duration and every shot's start time, so accumulating them gives the episode timeline. Zero extra requests, zero vendor coupling, and you control segmentation — one line of dialogue per cue, which is exactly the rhythm short drama wants.
  4. The key insight is that path one does not free you from path two: API timestamps are segment-relative, so you still add the shot's offset within the episode. Since you must write the alignment code anyway, make it the single source of truth.
  5. The implementation has one rule: the subtitle cursor and the shot cursor share one origin and advance together. Add a check that every cue falls inside its own shot — overflow raises no error, it just floats the previous shot's line over the next shot's picture.
  6. Expect the follow-up: what about karaoke-style word-level subtitles? That genuinely requires word-level timestamps from the API. Treat it as an optional enhancement over a local-alignment main path, degrading to sentence level when word data is unavailable.

  答题要点

  • 两条来源：接口返回的时间戳，以及由音频真实时长本地累加对齐
  • 接口那条的代价是多一次请求、时间戳只相对单段音频、字段结构跟厂商绑定
  • 本地对齐零额外请求零厂商依赖，断句按台词行控制，符合短剧节奏
  • 即使用接口时间戳也仍要自己加上这一镜在整集里的偏移，所以本地对齐代码无论如何都得写
  • 实现要点是字幕游标与镜头游标共用同一原点，并自检每条字幕是否落在它所属的镜头内

  Key points

  • Two sources: timestamps returned by the API, and local alignment accumulated from measured audio durations
  • The API path costs an extra request, gives segment-relative timestamps, and couples you to one vendor's field structure
  • Local alignment needs no extra request and no vendor coupling, and lets you segment per line of dialogue
  • Even with API timestamps you must add each shot's offset within the episode, so the alignment code is unavoidable anyway
  • The implementation rule is one shared origin for the subtitle and shot cursors, plus a check that each cue stays inside its own shot