Skip to main content

BookKeeper Proposals

This page describes a proposed BookKeeper Proposal (BP) process for proposing a major change to BookKeeper.

Process

What is considered a "major change" that needs a BP?

Any of the following should be considered a major change:

  • Any major new feature, subsystem, or piece of functionality
  • Any change that impacts the public interfaces of the project
  • Any change that impacts developer workflow of the project

All the following are public interfaces that people build around:

  • Binary log format
  • The network protocol and api behavior
  • Configuration, especially client configuration
  • Monitoring/Stats provider
  • Command line tools and arguments

What should be included in a BP?

A BP should contain the following sections:

  • Motivation: describe the problem to be solved
  • Proposed Change: describe the new thing you want to do. This may be fairly extensive and have large subsections of its own. Or it may be a few sentences, depending on the scope of the change.
  • New or Changed Public Interfaces: impact to any of the "compatibility commitments" described above. We want to call these out in particular so everyone thinks about them.
  • Migration Plan and Compatibility: if this feature requires additional support for a no-downtime upgrade describe how that will work
  • Rejected Alternatives: What are the other alternatives you considered and why are they worse? The goal of this section is to help people understand why this is the best solution now, and also to prevent churn in the future when old alternatives are reconsidered.

Who should initiate the BP?

Anyone can initiate a BP but you shouldn't do it unless you have an intention of getting the work done to implement it (otherwise it is silly).

How to make a BP?

Here is the process for making a BP:

  1. Create an issue BP-<number>: [capation of bookkeeper proposal]. E.g. BP-1: 64 bits ledger id support.
    • Take the next available BP number from this page.
    • Write a brief description about what BP is for in this issue. This issue will be the master issue for tracking the status of this BP and its implementations. All the implementations of this BP should be listed and linked to this master issues.
  2. Write the proposal for this BP. There are two ways to write a bookkeeper proposal. You can choose to write a BP using markdown, or write a BP using Google Doc.
    - Markdown
    - Make a copy of the [BP-Template](https://github.com/apache/bookkeeper/tree/master/site/bps/BP-template.md). Name the BP file as `BP-<number>-[caption-of-proposal].md`.
    ```shell
    $ cp site/bps/BP-template.md site/bps/BP-xyz-capation-of-proposal.md
    ```
    - Fill the sections listed in the BP template.
    - issue: replace `<issue-number>` with the issue number.
    - state: "Under Discussion"
    - release: leave the release to `N/A`. you can only mark a release after a BP is implemented.
    - Google Doc
    - Make a copy of the [BP-Template](https://docs.google.com/document/d/1DsmH54LoohgwqnEjESPQNtIYxxcOy2rwonZ_TJCwws0). Name the BP file as `BP-<number>-[caption-of-proposal]`.
    - Fill the sections listed in the BP template.
  3. Send a PR for this BP. Following the instructions in the pull request template.
    • add BP label to this PR
    • attach the google doc link in the PR description if the BP is written in google doc
    • don't associate this PR with any release or milestone
    • edit site/community/bookkeeper-proposals.md:
      • bump the next bp number
      • add this BP to Inprogress section
  4. You can tag committers on this RP for reviewers, or start a [DISCUSS] thread on Apache mailing list. If you are sending an email, please make sure that the subject of the thread is of the format [DISCUSS] BP-<number>: capation of bookkeeper proposal.
  5. Once the BP is finalized, reviewed and approved by committers, the BP is accepted. The criteria for acceptance is lazy majority.
    1. Committers merge the PR after a BP is accepted. The development for this BP moves forward with implementations. The BP should be updated if there is anything changed during implementing it.
    2. After all the implementations for a given BP are completed, a new PR should be sent for changing the state of a BP:
      • state: "Adopted"
      • release: set to the release that includes this BP.
      • moving the BP from Inprogress to Adopted.
    3. The final PR for changing BP state will be used as the criteria for marking a BP as completed.
  6. If a BP is failed or rejected:
    1. Update the PR to change the state of a BP
      • state: "Discarded"
      • add a paragraph at the first paragraph of this BP for describing the reasons.
      • moving the BP from Inprogress to Discarded.
    2. Once the PR is updated, committers can merge this proposal PR and close the master issue of this BP.

All Proposals

This section lists all the bookkeeper proposals made to BookKeeper.

Next Proposal Number: 43

Inprogress

ProposalState
BP-4 - BookKeeper Lifecycle ManagementDraft
BP-8 - Queue based auto rereplicatorDraft
BP-12 - Improve documentationAccepted
BP-14 Relax durabilityAccepted
BP-16: Thin Client - Remove direct metadata storage access from clientsDraft
BP-18: LedgerType, Flags and StorageHintsAccepted
BP-26: Move distributedlog library as part of bookkeeperAccepted
BP-27: New BookKeeper CLIAccepted
BP-28: use etcd as metadata storeAccepted
BP-29: Metadata API moduleAccepted
BP-30: BookKeeper Table ServiceAccepted
BP-31: BookKeeper Durability AnchorAccepted
BP-32: Advisory (optimistic) write closeAccepted
BP-33: Move releasing docker images out of main repoDraft
BP-34: Cluster Metadata CheckerAccepted
BP-35: 128 bits supportAccepted
BP-36: Stats documentation annotationAccepted
BP-37: Improve configuration management for better documentationAccepted
BP-41: Separate BookieId from Separate BookieId from Bookie Network AddressAccepted
BP-42: New Client API - list ledgersAccepted
BP-43: Migration to gradleDraft

Adopted

ProposalRelease
BP-1 - 64 bits ledger id support4.5.0
BP-2 - Resource aware data placement4.5.0
BP-3 - Security support4.5.0
BP-5 - Allow reads outside the LAC Protocol4.5.0
BP-6 - Use separate log for compaction4.6.0
BP-9 - Github issues for Issue Tracking4.5.0
BP-10 - Official Bookkeeper Docker Image4.5.0
BP-11 - Move website/documentation to Jekyll based site4.5.0
BP-13 - Time Based Release Plan4.6.0
BP-15 - New CreateLedger API4.6.0
BP-17 - Define BookKeeper public http endpoints4.6.0
BP-20: Github workflow for bookkeeper proposals4.7.0
BP-25: Move checksum to proto4.7.0
BP-38: Publish Bookie Service Info on Metadata Service4.11.0

Discarded

ProposalReason
BP-7 - Explicit LAC on addEntryNot A Problem
BP-21: New API close inconsistenciesNot A Problem
BP-22: Separate closing ledgers from opening ledgersNot A Problem