Topics, Partitions, Offsets and Records

The anatomy of the log

A record (message) has an optional key, a value, a timestamp and optional headers. A topic is split into partitions; each partition is an ordered, append-only sequence of records, and every record gets a sequential number in its partition called its offset. Ordering is guaranteed only within a partition, not across the whole topic. Partitions are the unit of parallelism: more partitions let more consumers read in parallel and spread data across brokers.

The log picture

Each partition grows at the end. Offsets never change, and a consumer remembers the next offset it will read.

topic "orders" (3 partitions)

partition 0:  [0][1][2][3][4]  -> next write goes to offset 5
partition 1:  [0][1][2]
partition 2:  [0][1][2][3]

consumer A reads partition 0 at offset 3; consumer B reads partition 1 at offset 1
Ordering is guaranteed inside each partition only.

Offsets are per partition

Offset 7 in partition 0 and offset 7 in partition 1 are different records. Always think of a position as the pair (partition, offset).

Quick check: Where is record ordering guaranteed?

• Within a single partition
• Across all partitions of a topic
• Across all topics
• Nowhere