Release Notes - Bookkeeper - Version 4.4.0

Sub-task

• [BOOKKEEPER-438] - Move ledger id generation out of LedgerManager
• [BOOKKEEPER-634] - Provide admin tool to rename bookie identifier in ledger metadata
• [BOOKKEEPER-796] - Make bookkeeper client use reconnectable zookeeper wrapper
• [BOOKKEEPER-837] - UpdateLedgerOp - Replace AbstractFuture with SettableFuture
• [BOOKKEEPER-855] - handle session expire event in bookie
• [BOOKKEEPER-867] - New Client API to allow applications pass-in EntryId.

Bug

• [BOOKKEEPER-578] - LedgerCacheImpl is reserving 1/3 of Heap size but allocates NonHeap memory
• [BOOKKEEPER-594] - AutoRecovery shutting down on SyncDisconnected
• [BOOKKEEPER-665] - BK client should not try to read entries from non-available bookies
• [BOOKKEEPER-695] - Some entry logs are not removed from the bookie storage
• [BOOKKEEPER-769] - Remove hedwig from source tree
• [BOOKKEEPER-793] - Move to java 7
• [BOOKKEEPER-794] - BookkeeperProtocol.Response.status is completely ignored
• [BOOKKEEPER-795] - Race condition causes writes to hang if ledger is fenced
• [BOOKKEEPER-797] - IllegalArgumentException when calling CodahaleOpStatsLogger#toOpStatsData()
• [BOOKKEEPER-799] - Distribution schedule coverage sets don't take gaps in response lists into account when writequorum > ackquorum
• [BOOKKEEPER-800] - Expose whether a ledger is closed or not
• [BOOKKEEPER-801] - Bookkeeper client tutorial
• [BOOKKEEPER-802] - Bookkeeper protocol documentation
• [BOOKKEEPER-803] - Guide for making a replicated log out of ledgers
• [BOOKKEEPER-804] - Client program is not terminated when using openLedgerNoRecovery
• [BOOKKEEPER-805] - NullPointException in bookie server when using twitter-ostrich-provider
• [BOOKKEEPER-809] - Wrong metric on LedgerDeleteOp and LedgerOpenOp
• [BOOKKEEPER-813] - BookieShell doesn't find index directory
• [BOOKKEEPER-814] - clean up temp files that generated by test cases.
• [BOOKKEEPER-815] - Ledger fence state is lost when the ledger file is evicted
• [BOOKKEEPER-821] - Failing to write lastId to ledger directories should not fail startup of bookies
• [BOOKKEEPER-823] - Clean up temp files created by hedwig tests
• [BOOKKEEPER-828] - Script for updating docs on website from master branch
• [BOOKKEEPER-831] - Outdated links in tutorial
• [BOOKKEEPER-833] - EntryLogId and EntryLogLimit should not be larger than Integer.MAX_VALUE
• [BOOKKEEPER-834] - test case error in test class TestDiskChecker
• [BOOKKEEPER-835] - Update copyright for 2015 on all active branches
• [BOOKKEEPER-838] - ForceWriteThread::run() leaks “logFile.close()” when interrupt comes
• [BOOKKEEPER-839] - AuditorPeriodicCheckTest timeout
• [BOOKKEEPER-840] - Deadlock on flushLock on compaction
• [BOOKKEEPER-848] - Use volatile for lastAddConfirmed
• [BOOKKEEPER-854] - NPE on InterleavedLedgerStorage.onRotateEntryLog
• [BOOKKEEPER-858] - Fix broken links and typos in bookkeeper documents
• [BOOKKEEPER-863] - Potential resource leak with unclosed LedgerManager in BookieShell
• [BOOKKEEPER-883] - Test timeout in bookkeeper-benchmark
• [BOOKKEEPER-890] - Concurrent modification exception when removing listener in Bookkeeper ZK ledger manager
• [BOOKKEEPER-891] - Read entries failure should trigger callback only once
• [BOOKKEEPER-893] - bookie exited with status 0 on journal I/O exception
• [BOOKKEEPER-897] - Fix findbugs warnings and missing apache license header
• [BOOKKEEPER-898] - Underreplication doesn't get triggered when a read only bookie is shut down
• [BOOKKEEPER-899] - Bookie should return to read-write mode once the disk usage drops before the threshold
• [BOOKKEEPER-900] - read only bookie runs replicator and does not release the under replicated lock after failing
• [BOOKKEEPER-902] - Test failures in EntryLogTest
• [BOOKKEEPER-904] - test BookieInitializationTest.testDuplicateBookieServerStartup fails on non-english machines
• [BOOKKEEPER-909] - ZooKeeper of LocalBookkeeper should use the correct tickTime
• [BOOKKEEPER-910] - In LocalBookkeeper, Zookeeper server and client use different host addresses
• [BOOKKEEPER-911] - Fix TestReplicationWorker test failures
• [BOOKKEEPER-913] - Fix flakiness in TestBackwardCompat
• [BOOKKEEPER-914] - ReadOnlyBookieTest.testBookieShouldTurnWritableFromReadOnly is intermettently failing
• [BOOKKEEPER-917] - LocalBookKeeperTest seems to be silently failing
• [BOOKKEEPER-919] - Auditor is sometimes marking as failed a bookie switching from available to read-only mode
• [BOOKKEEPER-921] - Typo in LocalBookkeeper: Use InetAddress.getHostAddress instead of InetAddress
• [BOOKKEEPER-925] - Fix FindBugs discovered issues in master
• [BOOKKEEPER-926] - Compacted entries are not properly synced before updating index

Documentation

• [BOOKKEEPER-870] - Change the default value for bookie settings.

Improvement

• [BOOKKEEPER-537] - Handling session expire event
• [BOOKKEEPER-687] - Use static final Logger for hedwig related modules
• [BOOKKEEPER-760] - Don't close PCBC proactively if bookies disappeared from zookeeper znodes.
• [BOOKKEEPER-810] - Allow to configure TCP connect timeout
• [BOOKKEEPER-811] - Recovery tool doesn't remove cookie after recovering one bookie
• [BOOKKEEPER-820] - print out fi.isFenced() in BookieShell
• [BOOKKEEPER-827] - change throttle in GarbageCollector to use either "by entry" or "by byte"
• [BOOKKEEPER-830] - Documentation has no structure
• [BOOKKEEPER-832] - Allow starting bookie in ReadOnly mode
• [BOOKKEEPER-836] - disable compaction when disk becomes full, otherwise compaction will fill up disk quickly
• [BOOKKEEPER-841] - Bookie should calculate ledgers map writing a new entry log file
• [BOOKKEEPER-844] - Add more metrics about latency and bytes characteristics on bookie operations
• [BOOKKEEPER-849] - Collect stats with sub-milliseconds precision
• [BOOKKEEPER-850] - Use nanoseconds to calculate poll timeout when doing group commit
• [BOOKKEEPER-851] - Configurable LedgerStorageImplementation
• [BOOKKEEPER-862] - Add tracing and stats to OrderedSafeExecutor for debugging slow tasks
• [BOOKKEEPER-866] - Fix compile issue when Updating junit to latest release version( 4.12) in the test of Bookkeeper-server.
• [BOOKKEEPER-877] - Script for generating patch for reviews
• [BOOKKEEPER-880] - Make LedgerHandle implement AutoCloseable
• [BOOKKEEPER-885] - Script to merge github pull request
• [BOOKKEEPER-886] - Allow to disable ledgers operation throttling
• [BOOKKEEPER-888] - Dispatch individual callbacks from journal in different threads
• [BOOKKEEPER-889] - BookKeeper client should try not to use bookies with errors/timeouts when forming a new ensemble
• [BOOKKEEPER-894] - Read ledger entries from the bookie shell
• [BOOKKEEPER-895] - bookies should not retain ledgers which no longer belong to them
• [BOOKKEEPER-920] - Extend bk-merge-pr.py to add more info to Jira ticket when merging

New Feature

• [BOOKKEEPER-879] - Record ledger creation time
• [BOOKKEEPER-901] - Add an authentication framework

Task

• [BOOKKEEPER-789] - Update README to reflect bookkeeper modules
• [BOOKKEEPER-790] - Add JNA license in NOTICE files
• [BOOKKEEPER-868] - Add ADD_ENTRY quorum timeout

Test

• [BOOKKEEPER-846] - TestLedgerChecker times out
• [BOOKKEEPER-892] - Add a sanity test to help identify bookie nodes with problems that prevent writes but is still registered in ZK