Deploying BookKeeper on DC/OS

DC/OS (the DataCenter Operating System) is a distributed operating system used for deploying and managing applications and systems on Apache Mesos. DC/OS is an open-source tool created and maintained by Mesosphere.

BookKeeper is available as a DC/OS package from the Mesosphere DC/OS Universe.

Prerequisites

In order to run BookKeeper on DC/OS, you will need:

• DC/OS version 1.8 or higher
• A DC/OS cluster with at least three nodes
• The DC/OS CLI tool installed

Each node in your DC/OS-managed Mesos cluster must have at least:

• 1 CPU
• 1 GB of memory
• 10 GB of total persistent disk storage

Installing BookKeeper

$ dcos package install bookkeeper --yes

This command will:

• Install the bookkeeper subcommand for the dcos CLI tool
• Start a single bookie on the Mesos cluster with the default configuration

The bookie that is automatically started up uses the host mode of the network and by default exports the service at agent_ip:3181.

If you run dcos package install bookkeeper without setting the --yes flag, the install will run in interactive mode. For more information on the package install command, see the DC/OS docs.

Services

To watch BookKeeper start up, click on the Services tab in the DC/OS user interface and you should see the bookkeeper package listed:

Tasks

To see which tasks have started, click on the bookkeeper service and you'll see an interface that looks like this;

Scaling BookKeeper

Once the first bookie has started up, you can click on the Scale tab to scale up your BookKeeper ensemble by adding more bookies (or scale down the ensemble by removing bookies).

ZooKeeper Exhibitor

ZooKeeper contains the information for all bookies in the ensemble. When deployed on DC/OS, BookKeeper uses a ZooKeeper instance provided by DC/OS. You can access a visual UI for ZooKeeper using Exhibitor, which is available at http://master.dcos/exhibitor.

You should see a listing of IP/host information for all bookies under the messaging/bookkeeper/ledgers/available node.

Client connections

To connect to bookies running on DC/OS using clients running within your Mesos cluster, you need to specify the ZooKeeper connection string for DC/OS's ZooKeeper cluster:

master.mesos:2181

This is the only ZooKeeper host/port you need to include in your connection string. Here's an example using the Java client:

BookKeeper bkClient = new BookKeeper("master.mesos:2181");

If you're connecting using a client running outside your Mesos cluster, you need to supply the public-facing connection string for your DC/OS ZooKeeper cluster.

Configuring BookKeeper

By default, the bookkeeper package will start up a BookKeeper ensemble consisting of one bookie with one CPU, 1 GB of memory, and a 70 MB persistent volume.

You can supply a non-default configuration when installing the package using a JSON file. Here's an example command:

$ dcos package install bookkeeper \
  --options=/path/to/config.json

You can then fetch the current configuration for BookKeeper at any time using the package describe command:

$ dcos package describe bookkeeper \
  --config

Available parameters

Not all configurable parameters for BookKeeper are available for BookKeeper on DC/OS. Only the parameters show in the table below are available.

Param	Type	Description	Default
name	String	The name of the DC/OS service.	bookkeeper
cpus	Integer	The number of CPU shares to allocate to each bookie. The minimum is 1.	1
instances	Integer	The number of bookies top run. The minimum is 1.	1
mem	Number	The memory, in MB, to allocate to each BookKeeper task	1024.0 (1 GB)
volume_size	Number	The persistent volume size, in MB	70
zk_client	String	The connection string for the ZooKeeper client instance	master.mesos:2181
service_port	Integer	The BookKeeper export service port, using PORT0 in Marathon	3181

Example JSON configuration

Here's an example JSON configuration object for BookKeeper on DC/OS:

{
  "instances": 5,
  "cpus": 3,
  "mem": 2048.0,
  "volume_size": 250
}

If that configuration were stored in a file called bk-config.json, you could apply that configuration upon installating the BookKeeper package using this command:

$ dcos package install bookkeeper \
  --options=./bk-config.json

Uninstalling BookKeeper

You can shut down and uninstall the bookkeeper from DC/OS at any time using the package uninstall command:

$ dcos package uninstall bookkeeper
Uninstalled package [bookkeeper] version [4.11.1]
Thank you for using bookkeeper.