# Failures, Retries and Dead-Letter Topics — Apache Kafka: Event Streaming from Basics to Production

Source: https://www.geekswithgeeks.com/en/kafka/kc-failures-dlq

> Handle poison messages without blocking the partition.

## One bad record must not stop the line

A partition is processed in order, so a record that always fails (a **poison message**: bad data, unexpected schema) can block everything behind it if you retry forever. Separate **transient** failures (a database timeout, which deserve a few retries with backoff) from **permanent** ones (malformed data, which retries cannot fix). Send permanent failures to a **dead-letter topic** (DLT) together with the error reason and original coordinates (topic, partition, offset), commit the offset, and continue. Monitor the DLT size and have a process to inspect and replay its records after fixing the cause. Where ordering matters, retry in place rather than moving the record elsewhere.

## Routing a failure to a dead-letter topic (illustrative)

`TransientError` is retried by your code; anything else is considered permanent and parked with context in `orders.dlt`.

```python
try:
    handle(msg)
except TransientError:
    raise                                  # let the retry policy handle it
except Exception as exc:                   # permanent: park it, keep going
    producer.produce("orders.dlt", key=msg.key(), value=msg.value(),
                     headers=[("error", str(exc).encode()),
                              ("src", f"{msg.topic()}[{msg.partition()}]@{msg.offset()}".encode())])
commit(msg)
```

**Quiz:** Why use a dead-letter topic?

- [ ] To speed up the broker
- [x] To park permanently failing records so the partition keeps moving
- [ ] To delete all errors silently
- [ ] To increase partitions

*Answer:* To park permanently failing records so the partition keeps moving. Parked records can be inspected and replayed without blocking healthy traffic.