Architecture: Brokers, Topics and Partitions

Harry · 13 Sep 2026 · 1 views

Brokers

A broker is a Kafka server process. A cluster runs several brokers; every topic partition is owned by one broker and replicated (leader/follower) across others for fault tolerance.

Topics and Partitions

A topic is a named stream of records. It is split into partitions: each partition is an ordered, immutable log. Keys route records to partitions, and partition count drives parallelism.

Offsets

Each record in a partition has a sequential offset. Consumers track their read position per partition, enabling replay from any offset.

Replication and ISR

Each partition keeps a leader and in-sync replicas (ISR). Writes go to the leader; followers replicate and report progress. If a leader fails, a follower takes over.

ZooKeeper / KRaft

Classic Kafka used ZooKeeper for cluster metadata. Kafka 3.3+ ships KRaft mode (built-in controller), removing ZooKeeper entirely for new clusters.

Key Points

  • Brokers host partitions; partitions are ordered logs.
  • Keys route records; offsets track positions.
  • Leader plus ISR replicas guarantee durability.
  • KRaft replaces ZooKeeper on modern clusters.

Kafka pipeline: producers to topics to consumers

Share this post:
What Is Apache Kafka? Installing Kafka

Comments (0)

Please login or register to comment.