面试题库

30 天从前端工程师到 Agent 工程师

D9 Redis Streams 消息总线：XADD/XREADGROUP/XACK/XAUTOCLAIM、consumer group、毒消息

• Redis Streams 的 consumer group 是怎么工作的？为什么它既能做工作队列又能做发布订阅？How does a Redis Streams consumer group work, and why can it serve both as a work queue and as pub/sub?

  国内高频海外高频基础#message-bus#redis-streams

  分析过程 · 先想清楚再作答

  1. 这题是概念题，区分度在于你有没有把「组」和「消费者」两层分清。只答「多个消费者一起消费」会被追着问「那同一条消息会不会被消费两次」，而这正是两层的区别所在。
  2. 先给两层结构：流本身只增不减，组挂在流上、维护一个读游标和一份 pending 清单，消费者挂在组上、只是组内的一个名字。同一个组内的消费者分摊消息（一条只进一个人），不同的组各自都能读到全量——工作队列和发布订阅就是这一个数据结构的两种用法。
  3. 接着点出 pending 清单（PEL）记了哪三件事：这条消息归哪个消费者、被投递过几次、最后一次投递在什么时刻。这三列分别对应「谁在处理」「要不要判成毒消息」「能不能被别人接手」，答出来就说明你真的读过文档而不只是抄过示例。
  4. 结论要落到分配规则上：组把消息分给谁，完全取决于谁先来问，没有任何亲和性。所以 consumer group 天然不保证「同一个用户的多条消息按顺序被同一个人处理」——这一句是把话题引向自己准备好的深水区。
  5. 可以预期的追问一：消费者的名字该怎么取？答：随机名会让进程重启后老名字下的未确认消息变成孤儿，只能靠 XAUTOCLAIM 捡回来，所以要么用有状态部署给的稳定序号，要么就必须依赖 XAUTOCLAIM 兜底，并定期用 XGROUP DELCONSUMER 清理不会再回来的名字。
  6. 可以预期的追问二：怎么保住同一个用户的顺序？答：在总线之上做分片——把用户 id 哈希到固定数量的分片，每个分片同一时刻只由一个消费者持有，顺序就回来了。消费组本身解决不了这件事。

  How to reason about it · think before answering

  1. This is a concept question; the discriminator is whether you separate the group layer from the consumer layer. Saying only 'several consumers read together' invites 'so is a message processed twice?' — and that is exactly what the two layers settle.
  2. Give the structure: the stream is append-only; a group sits on the stream and owns a read cursor plus a pending list; a consumer is just a name inside a group. Consumers in one group share the messages (each message goes to exactly one of them), while separate groups each see the full stream — one data structure, both a work queue and pub/sub.
  3. Then name the three things the pending entries list records: which consumer owns the message, how many times it has been delivered, and when it was last delivered. Those map to 'who is working on it', 'is it poison yet' and 'can someone else take over' — knowing them signals you read the docs, not just a snippet.
  4. Land on the dispatch rule: a group hands a message to whoever asks first, with no affinity at all. So a consumer group does not keep multiple messages from the same user in order on the same worker — say this yourself and you steer into ground you have prepared.
  5. Expect: how do you name consumers? Random names orphan the unacked messages of the previous name after a restart, recoverable only via XAUTOCLAIM. Either use stable ordinals from a stateful deployment, or rely on XAUTOCLAIM and periodically prune dead names with XGROUP DELCONSUMER.
  6. Expect: how do you preserve per-user order? Shard above the bus — hash the user id onto a fixed number of shards and let one consumer own a shard at a time. The consumer group cannot do this for you.

  答题要点

  • 流只增不减；组挂在流上，维护读游标和 pending 清单；消费者是组内的一个名字
  • 同组内消息被分摊（一条只进一个消费者），不同组各自拿到全量，所以同一个结构同时支持工作队列和发布订阅
  • pending 清单记三件事：归属的消费者、投递次数、最后一次投递时刻，分别用于接手、毒消息判定和超时检测
  • 分配没有亲和性，谁先来问给谁，所以不保证同一个用户的多条消息顺序，要在总线之上做分片
  • 消费者名字随机会在重启后留下孤儿消息，要么名字稳定，要么依赖 XAUTOCLAIM 并清理死名字

  Key points

  • The stream is append-only; a group holds a read cursor and a pending list; a consumer is a name within a group
  • Within a group messages are split (one message, one consumer); separate groups each get everything, so one structure covers both work queue and pub/sub
  • The pending list records owner, delivery count and last-delivery time — used for takeover, poison detection and timeouts
  • Dispatch has no affinity, so per-user ordering is not guaranteed and needs sharding above the bus
  • Random consumer names orphan unacked messages after a restart; use stable names or rely on XAUTOCLAIM plus XGROUP DELCONSUMER cleanup