A reference guide to the command-line tools that you can use to administer BookKeeper


bookkeeper

Manages bookies.

Environment variables

Environment variable Description Default
BOOKIE_LOG_CONF

The Log4j configuration file.

${bookkeeperHome}/bookkeeper-server/conf/log4j.properties
BOOKIE_CONF

The configuration file for the bookie.

${bookkeeperHome}/bookkeeper-server/conf/bk_server.conf
BOOKIE_EXTRA_CLASSPATH

Extra paths to add to BookKeeper’s classpath.

ENTRY_FORMATTER_CLASS

The entry formatter class used to format entries.

BOOKIE_PID_DIR

The directory where the bookie server PID file is stored.

BOOKIE_STOP_TIMEOUT

The wait time before forcefully killing the bookie server instance if stopping it is not successful.

Commands

bookie

Starts up a bookie.

Usage
$ bookkeeper-server/bin/bookkeeper bookie 

localbookie

Starts up an ensemble of N bookies in a single JVM process. Typically used for local experimentation and development.

Usage
$ bookkeeper-server/bin/bookkeeper localbookie N

autorecovery

Runs the autorecovery service daemon.

Usage
$ bookkeeper-server/bin/bookkeeper autorecovery 

upgrade

Upgrades the bookie’s filesystem.

Usage
$ bookkeeper-server/bin/bookkeeper upgrade 

shell

Runs the bookie’s shell for admin commands.

Usage
$ bookkeeper-server/bin/bookkeeper shell 

help

Displays the help message for the bookkeeper tool.

Usage
$ bookkeeper-server/bin/bookkeeper help 

The BookKeeper shell

autorecovery

Enable or disable autorecovery in the cluster.

Usage
$ bookkeeper-server/bin/bookkeeper shell autorecovery \
  <options>
Options
Flag Description
-enable Enable autorecovery of underreplicated ledgers
-disable Disable autorecovery of underreplicated ledgers


bookieFormat

Format the current server contents.

Usage
$ bookkeeper-server/bin/bookkeeper shell bookieFormat \
  <options>
Options
Flag Description
-nonInteractive Whether to confirm if old data exists.
-force If [nonInteractive] is specified, then whether to force delete the old data without prompt..?
-deleteCookie Delete its cookie on zookeeper


bookieinfo

Retrieve bookie info such as free and total disk space.

Usage
$ bookkeeper-server/bin/bookkeeper shell bookieinfo


bookiesanity

Sanity test for local bookie. Create ledger and write/read entries on the local bookie.

Usage
$ bookkeeper-server/bin/bookkeeper shell bookiesanity \
  <options>
Options
Flag Description
-entries N Total entries to be added for the test (default 10)
-timeout N Timeout for write/read operations in seconds (default 1)


decommissionbookie

Force trigger the Audittask and make sure all the ledgers stored in the decommissioning bookie are replicated.

Usage
$ bookkeeper-server/bin/bookkeeper shell decommissionbookie


deleteledger

Delete a ledger

Usage
$ bookkeeper-server/bin/bookkeeper shell deleteledger \
  <options>
Options
Flag Description
-ledgerid N Ledger ID
-force Whether to force delete the Ledger without prompt..?


expandstorage

Add new empty ledger/index directories. Update the directories info in the conf file before running the command.

Usage
$ bookkeeper-server/bin/bookkeeper shell expandstorage


help

Displays the help message.

Usage
$ bookkeeper-server/bin/bookkeeper shell help


lastmark

Print last log marker.

Usage
$ bookkeeper-server/bin/bookkeeper shell lastmark


ledger

Dump ledger index entries into readable format.

Usage
$ bookkeeper-server/bin/bookkeeper shell ledger \
  <options>
Options
Flag Description
-m LEDGER_ID Print meta information


ledgermetadata

Print the metadata for a ledger.

Usage
$ bookkeeper-server/bin/bookkeeper shell ledgermetadata \
  <options>
Options
Flag Description
-ledgerid N Ledger ID


listbookies

List the bookies, which are running as either readwrite or readonly mode.

Usage
$ bookkeeper-server/bin/bookkeeper shell listbookies \
  <options>
Options
Flag Description
-readwrite Print readwrite bookies
-readonly Print readonly bookies
-hostnames Also print hostname of the bookie


listfilesondisc

List the files in JournalDirectory/LedgerDirectories/IndexDirectories.

Usage
$ bookkeeper-server/bin/bookkeeper shell listfilesondisc \
  <options>
Options
Flag Description
-journal Print list of journal files
-entrylog Print list of entryLog files
-index Print list of index files


listledgers

List all ledgers in the cluster (this may take a long time).

Usage
$ bookkeeper-server/bin/bookkeeper shell listledgers \
  <options>
Options
Flag Description
-meta Print metadata


listunderreplicated

List ledgers marked as underreplicated, with optional options to specify missing replica (BookieId) and to exclude missing replica.

Usage
$ bookkeeper-server/bin/bookkeeper shell listunderreplicated \
  <options>
Options
Flag Description
-missingreplica N Bookie Id of missing replica
-excludingmissingreplica N Bookie Id of missing replica to ignore


metaformat

Format Bookkeeper metadata in Zookeeper.

Usage
$ bookkeeper-server/bin/bookkeeper shell metaformat \
  <options>
Options
Flag Description
-nonInteractive Whether to confirm if old data exists..?
-force If [nonInteractive] is specified, then whether to force delete the old data without prompt.


lostbookierecoverydelay

Setter and Getter for LostBookieRecoveryDelay value (in seconds) in Zookeeper.

Usage
$ bookkeeper-server/bin/bookkeeper shell lostbookierecoverydelay \
  <options>
Options
Flag Description
-get Get LostBookieRecoveryDelay value (in seconds)
-set N Set LostBookieRecoveryDelay value (in seconds)


readjournal

Scan a journal file and format the entries into readable format.

Usage
$ bookkeeper-server/bin/bookkeeper shell readjournal \
  <options>
Options
Flag Description
-msg JOURNAL_ID|JOURNAL_FILENAME Print message body
-dir Journal directory (needed if more than one journal configured)


readledger

Read a range of entries from a ledger.

Usage
$ bookkeeper-server/bin/bookkeeper shell readledger \
  <ledger_id> [<start_entry_id> [<end_entry_id>]]


readlog

Scan an entry file and format the entries into readable format.

Usage
$ bookkeeper-server/bin/bookkeeper shell readlog \
  <entry_log_id | entry_log_file_name> \
  <options>
Options
Flag Description
-msg Print message body
-ledgerid N Ledger ID
-entryid N Entry ID
-startpos N Start Position
-endpos End Position


recover

Recover the ledger data for failed bookie.

Usage
$ bookkeeper-server/bin/bookkeeper shell recover \
  <bookieSrc> [<bookieDest>] \
  <options>
Options
Flag Description
-deleteCookie Delete cookie node for the bookie.


simpletest

Simple test to create a ledger and write entries to it.

Usage
$ bookkeeper-server/bin/bookkeeper shell simpletest \
  <options>
Options
Flag Description
-ensemble N Ensemble size (default 3)
-writeQuorum N Write quorum size (default 2)
ackQuorum N Ack quorum size (default 2)
-numEntries N Entries to write (default 1000)


triggeraudit

Force trigger the Audit by resetting the lostBookieRecoveryDelay.

Usage
$ bookkeeper-server/bin/bookkeeper shell triggeraudit


updatecookie

Update bookie id in cookie.

Usage
$ bookkeeper-server/bin/bookkeeper shell updatecookie \
  <options>
Options
Flag Description
-bookieId <hostname|ip> Bookie Id


updateledgers

Update bookie id in ledgers (this may take a long time).

Usage
$ bookkeeper-server/bin/bookkeeper shell updateledgers \
  <options>
Options
Flag Description
-bookieId <hostname|ip> Bookie Id
-updatespersec N Number of ledgers updating per second (default 5 per sec)
-limit N Maximum number of ledgers to update (default no limit)
-verbose Print status of the ledger updation (default false)
-printprogress N Print messages on every configured seconds if verbose turned on (default 10 secs)

An entry is a sequence of bytes (plus some metadata) written to a BookKeeper ledger. Entries are also known as records.

A ledger is a sequence of entries written to BookKeeper. Entries are written sequentially to ledgers and at most once, giving ledgers append-only semantics.

A bookie is an individual BookKeeper storage server.

Bookies store the content of ledgers and act as a distributed ensemble.

A subsystem that runs in the background on bookies to ensure that ledgers are fully replicated even if one bookie from the ensemble is down.

Striping is the process of distributing BookKeeper ledgers to sub-groups of bookies rather than to all bookies in a BookKeeper ensemble.

Striping is essential to ensuring fast performance.

A journal file stores BookKeeper transaction logs.

When a reader forces a ledger to close, preventing any further entries from being written to the ledger.

A record is a sequence of bytes (plus some metadata) written to a BookKeeper ledger. Records are also known as entries.