Lesson 9 / 32

Message Queues

Decoupling producers from consumers with async buffers.

Why a queue

A queue (Kafka, SQS, RabbitMQ) lets a fast producer hand off work and move on. Consumers process at their own pace, absorbing spikes and surviving restarts.

Delivery guarantees

Most queues offer at-least-once delivery, so consumers must be idempotent — processing the same message twice should be safe.

Add a dead-letter queue

Messages that fail repeatedly go to a DLQ instead of blocking the pipeline. You inspect and replay them later.

Quick check: Why must consumers of an at-least-once queue be idempotent?

  • Messages may arrive out of order
  • The same message may be delivered more than once
  • Messages can be lost
Answer

The same message may be delivered more than once — Retries and redeliveries mean duplicates; idempotent handling makes them harmless.