Package org.apache.bookkeeper.bookie

Class DefaultEntryLogger

java.lang.Object
org.apache.bookkeeper.bookie.DefaultEntryLogger
All Implemented Interfaces:
AutoCloseable, EntryLogger
Direct Known Subclasses:
ReadOnlyDefaultEntryLogger
public class DefaultEntryLogger extends Object implements EntryLogger
This class manages the writing of the bookkeeper entries. All the new entries are written to a common log. The LedgerCache will have pointers into files created by this class with offsets into the files to find the actual ledger entry. The entry log files created by this class are identified by a long.

  • Field Details

    • UNINITIALIZED_LOG_ID

      static final int UNINITIALIZED_LOG_ID
      See Also:

    • recentlyCreatedEntryLogsStatus

      final DefaultEntryLogger.RecentEntryLogsStatus recentlyCreatedEntryLogsStatus

    • entryLoggerAllocator

      final EntryLoggerAllocator entryLoggerAllocator

    • HEADER_CURRENT_VERSION

      static final int HEADER_CURRENT_VERSION
      See Also:

    • LOGFILE_HEADER_SIZE

      static final int LOGFILE_HEADER_SIZE
      The 1K block at the head of the entry logger file that contains the fingerprint and meta-data. 
       Header is composed of:
 Fingerprint: 4 bytes "BKLO"
 Log file HeaderVersion enum: 4 bytes
 Ledger map offset: 8 bytes
 Ledgers Count: 4 bytes
      See Also:

    • HEADER_VERSION_POSITION

      static final int HEADER_VERSION_POSITION
      See Also:

    • LEDGERS_MAP_OFFSET_POSITION

      static final int LEDGERS_MAP_OFFSET_POSITION
      See Also:

    • LEDGERS_MAP_HEADER_SIZE

      static final int LEDGERS_MAP_HEADER_SIZE
      Ledgers map is composed of multiple parts that can be split into separated entries. Each of them is composed of: 
       length: (4 bytes) [0-3]
 ledger id (-1): (8 bytes) [4 - 11]
 entry id: (8 bytes) [12-19]
 num ledgers stored in current metadata entry: (4 bytes) [20 - 23]
 ledger entries: sequence of (ledgerid, size) (8 + 8 bytes each) [24..]
      See Also:

    • LEDGERS_MAP_ENTRY_SIZE

      static final int LEDGERS_MAP_ENTRY_SIZE
      See Also:

    • LEDGERS_MAP_MAX_BATCH_SIZE

      static final int LEDGERS_MAP_MAX_BATCH_SIZE
      See Also:

    • INVALID_LID

      static final long INVALID_LID
      See Also:

    • LEDGERS_MAP_ENTRY_ID

      static final long LEDGERS_MAP_ENTRY_ID
      See Also:

    • MIN_SANE_ENTRY_SIZE

      static final int MIN_SANE_ENTRY_SIZE
      See Also:

    • MB

      static final long MB
      See Also:

    • conf

      final ServerConfiguration conf

  • Constructor Details

  • Method Details

    • getEntryLogManager

      EntryLogManager getEntryLogManager()

    • addListener

      void addListener(DefaultEntryLogger.EntryLogListener listener)

    • putInReadChannels

      public BufferedReadChannel putInReadChannels(long logId, BufferedReadChannel bc)
      Put the logId, bc pair in the map responsible for the current thread.
      Parameters:
      logId -
      bc -

    • removeFromChannelsAndClose

      public void removeFromChannelsAndClose(long logId)
      Remove all entries for this log file in each thread's cache.
      Parameters:
      logId -

    • getFromChannels

      public BufferedReadChannel getFromChannels(long logId)

    • getLeastUnflushedLogId

      long getLeastUnflushedLogId()

    • getFlushedLogIds

      public Set<Long> getFlushedLogIds()
      Description copied from interface: EntryLogger
      Get the log ids for the set of logs which have been completely flushed to disk. Only log ids in this set are considered for either compaction or garbage collection.
      Specified by:
      getFlushedLogIds in interface EntryLogger

    • getPreviousAllocatedEntryLogId

      long getPreviousAllocatedEntryLogId()

    • prepareSortedLedgerStorageCheckpoint

      void prepareSortedLedgerStorageCheckpoint(long numBytesFlushed) throws IOException
      Throws:
      IOException

    • prepareEntryMemTableFlush

      void prepareEntryMemTableFlush()

    • commitEntryMemTableFlush

      boolean commitEntryMemTableFlush() throws IOException
      Throws:
      IOException

    • getEntryLoggerAllocator

      EntryLoggerAllocator getEntryLoggerAllocator()
      get EntryLoggerAllocator, Just for tests.

    • removeEntryLog

      public boolean removeEntryLog(long entryLogId)
      Remove entry log.
      Specified by:
      removeEntryLog in interface EntryLogger
      Parameters:
      entryLogId - Entry Log File Id
      Returns:
      false if the entrylog doesn't exist.

    • clearCompactingLogId

      void clearCompactingLogId()

    • checkpoint

      void checkpoint() throws IOException
      Flushes all rotated log channels. After log channels are flushed, move leastUnflushedLogId ptr to current logId.
      Throws:
      IOException

    • flush

      public void flush() throws IOException
      Description copied from interface: EntryLogger
      Flush any outstanding writes to disk.
      Specified by:
      flush in interface EntryLogger
      Throws:
      IOException

    • addEntry

      long addEntry(long ledger, ByteBuffer entry) throws IOException
      Throws:
      IOException

    • addEntry

      long addEntry(long ledger, io.netty.buffer.ByteBuf entry, boolean rollLog) throws IOException
      Throws:
      IOException

    • addEntry

      public long addEntry(long ledger, io.netty.buffer.ByteBuf entry) throws IOException
      Description copied from interface: EntryLogger
      Add an entry for ledger ```ledgerId``` to the entrylog.
      Specified by:
      addEntry in interface EntryLogger
      Parameters:
      ledger - the ledger for which the entry is being added
      entry - the contents of the entry (this method does not take ownership of the refcount)
      Returns:
      the location in the entry log of the added entry
      Throws:
      IOException

    • logIdForOffset

      static long logIdForOffset(long offset)

    • posForOffset

      static long posForOffset(long location)

    • checkEntry

      void checkEntry(long ledgerId, long entryId, long location) throws DefaultEntryLogger.EntryLookupException, IOException
      Throws:
      DefaultEntryLogger.EntryLookupException
      IOException

    • readEntry

      public io.netty.buffer.ByteBuf readEntry(long ledgerId, long entryId, long entryLocation) throws IOException, Bookie.NoEntryException
      Description copied from interface: EntryLogger
      Read an entry from an entrylog location, and verify that is matches the expected ledger and entry ID.
      Specified by:
      readEntry in interface EntryLogger
      Parameters:
      ledgerId - the ledgerID to match
      entryId - the entryID to match
      entryLocation - the location from which to read the entry
      Returns:
      the entry
      Throws:
      IOException
      Bookie.NoEntryException

    • readEntry

      public io.netty.buffer.ByteBuf readEntry(long location) throws IOException, Bookie.NoEntryException
      Description copied from interface: EntryLogger
      Read an entry from an entrylog location.
      Specified by:
      readEntry in interface EntryLogger
      Parameters:
      location - the location from which to read the entry
      Returns:
      the entry
      Throws:
      IOException
      Bookie.NoEntryException

    • getChannelForLogId

      BufferedReadChannel getChannelForLogId(long entryLogId) throws IOException
      Throws:
      IOException

    • logExists

      public boolean logExists(long logId)
      Whether the log file exists or not.
      Specified by:
      logExists in interface EntryLogger

    • getEntryLogsSet

      public Set<Long> getEntryLogsSet() throws IOException
      Returns a set with the ids of all the entry log files.
      Throws:
      IOException

    • scanEntryLog

      public void scanEntryLog(long entryLogId, EntryLogScanner scanner) throws IOException
      Scan entry log.
      Specified by:
      scanEntryLog in interface EntryLogger
      Parameters:
      entryLogId - Entry Log Id
      scanner - Entry Log Scanner
      Throws:
      IOException

    • getEntryLogMetadata

      public EntryLogMetadata getEntryLogMetadata(long entryLogId, AbstractLogCompactor.Throttler throttler) throws IOException
      Description copied from interface: EntryLogger
      Retrieve metadata for the given entrylog ID. The metadata contains the size of the log, the size of the data in the log which is still active, and a list of all the ledgers contained in the log and the size of the data stored for each ledger.
      Specified by:
      getEntryLogMetadata in interface EntryLogger
      Throws:
      IOException

    • extractEntryLogMetadataFromIndex

      EntryLogMetadata extractEntryLogMetadataFromIndex(long entryLogId) throws IOException
      Throws:
      IOException

    • close

      public void close()
      Shutdown method to gracefully stop entry logger.
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface EntryLogger

    • getLedgerDirsManager

      protected LedgerDirsManager getLedgerDirsManager()

    • fileName2LogId

      static long fileName2LogId(String fileName)
      Convert log filename (hex format with suffix) to logId in long.

    • logId2HexString

      static String logId2HexString(long logId)
      Convert log Id to hex string.

    • newCompactionLog

      public CompactionEntryLog newCompactionLog(long logToCompact) throws IOException
      Description copied from interface: EntryLogger
      Create a new entrylog into which compacted entries can be added. There is a 1-1 mapping between logs that are being compacted and the log the compacted entries are written to.
      Specified by:
      newCompactionLog in interface EntryLogger
      Throws:
      IOException

    • incompleteCompactionLogs

      public Collection<CompactionEntryLog> incompleteCompactionLogs()
      Description copied from interface: EntryLogger
      Return a collection of all the compaction entry logs which have been compacted, but have not been cleaned up.
      Specified by:
      incompleteCompactionLogs in interface EntryLogger