Metadata Management

There are two kinds of metadata needs to be managed in BookKeeper: one is the list of available bookies, which is used to track server availability (ZooKeeper is designed naturally for this); while the other is ledger metadata, which could be handle by different kinds of key/value storages efficiently with CAS (Compare And Set) semantics.

Ledger metadata is handled by LedgerManager and can be plugged with various storage mediums.

Ledger Metadata Management

The operations on the metadata of a ledger are quite straightforward. They are:

How to choose a metadata storage medium for BookKeeper.

From the interface, several requirements need to met before choosing a metadata storage medium for BookKeeper:

ZooKeeper is the default implemention for BookKeeper metadata management, ZooKeeper holds data in memory and provides filesystem-like namespace and also meets all the above requirements. ZooKeeper could meet most of usages for BookKeeper. However, if you application needs to manage millions of ledgers, a more scalable solution would be HBase, which also meet the above requirements, but it more complicated to set up.