Stateless Brokers

In Tansu, brokers are stateless and equal:

• Brokers do not replicate storage, so there is no concept of an insync replica
• Every broker is the leader of each topic partition
• Every broker is the coordinator of every consumer group
• Every broker is the coordinator of every transaction

A client will issue to "find coordinator" request to the cluster asking for the node that is responsible for coordinating a consumer group. This results in a "ping pong" of coordinator requests during network issues, or while scaling the cluster, while the new coordinator is elected and that information is propagated to the cluster.

In Tansu when a request is made on a consumer group, the receiving broker optimistically executes the operation using its current state. At the end of the operational the new state is persisted to the storage engine. If the state was out of date, the storage engine responds with the current state and the broker executes the operation with the newer state. The storage engine continues to reject requests that weren't executed on the current state.

Brokers act without coordination (e.g., Raft, ZooKeeper) by optimistically applying operations on their current state. At the end of the operation the broker attempts to persist its current state. The storage engine rejects operations that were not performed on the latest state. If the operation is rejected by the storage engine, the broker reaplies the operation with the new current state and tries to persist again.

Each consumer group has its own indepent state.