Consumers and Consumer Groups

Consumers

A consumer reads records from partitions and tracks its offset in the group. It processes each record and (with manual commits) acknowledges progress.

Consumer Groups and Partition Assignment

Consumers with the same group.id share the topic: each partition is assigned to exactly one consumer in the group. Adding consumers rebalances partitions across them; more consumers scale out until partitions run out.

Group Rules

More consumers than partitions is wasteful - the extras stay idle. Group members track offsets in the __consumer_offsets topic.

Commit Strategies

enable.auto.commit=false   # manual control
consumer.commitSync()      # or commitAsync()

At-least-once (commit after processing) is the default mental model. Exactly-once requires transactions and read_committed isolation.

Resetting Offsets

bin/kafka-consumer-groups.sh --bootstrap-server localhost:9092 
  --group billing --topic orders --reset-offsets --to-earliest --execute

Key Points

• Groups divide partitions among members.
• Parallelism caps at partition count.
• Offset commits define delivery semantics.
• Reset tools rewind or advance position.