Package org.apache.bookkeeper.bookie

Class Journal

java.lang.Object

org.apache.bookkeeper.bookie.Journal

All Implemented Interfaces:

CheckpointSource

public class Journal
extends java.lang.Object
implements CheckpointSource

Provide journal related management.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	Journal.BufferedChannelBuilder	For testability.
static class	Journal.ForceWriteRequest	Token which represents the need to force a write to the Journal.
static interface	Journal.JournalIdFilter	Filter to pickup journals.
static interface	Journal.JournalScanner	Scanner used to scan a journal.
class	Journal.LastLogMark	Last Log Mark.
(package private) static class	Journal.QueueEntry	Journal Entry to Record.

Nested classes/interfaces inherited from interface org.apache.bookkeeper.bookie.CheckpointSource

CheckpointSource.Checkpoint

Field Summary

Fields

Modifier and Type	Field	Description
(package private) ServerConfiguration	conf
(package private) FileChannelProvider	fileChannelProvider
(package private) BatchedBlockingQueue<Journal.ForceWriteRequest>	forceWriteRequests
(package private) org.apache.bookkeeper.bookie.Journal.ForceWriteThread	forceWriteThread
(package private) java.io.File	journalDirectory
(package private) long	journalPreAllocSize
(package private) int	journalWriteBufferSize
(package private) static int	KB
(package private) int	maxBackupJournals
(package private) long	maxJournalSize
(package private) static long	MB
(package private) static int	PADDING_MASK
(package private) BatchedBlockingQueue<Journal.QueueEntry>	queue
(package private) boolean	running

Fields inherited from interface org.apache.bookkeeper.bookie.CheckpointSource

DEFAULT

Constructor Summary

Constructors

Constructor	Description
Journal(int journalIndex, java.io.File journalDirectory, ServerConfiguration conf, LedgerDirsManager ledgerDirsManager)
Journal(int journalIndex, java.io.File journalDirectory, ServerConfiguration conf, LedgerDirsManager ledgerDirsManager, StatsLogger statsLogger, io.netty.buffer.ByteBufAllocator allocator)
Journal(int journalIndex, java.io.File journalDirectory, ServerConfiguration conf, LedgerDirsManager ledgerDirsManager, StatsLogger statsLogger, io.netty.buffer.ByteBufAllocator allocator, JournalAliveListener journalAliveListener)

Method Summary

Modifier and Type	Method	Description
void	checkpointComplete(CheckpointSource.Checkpoint checkpoint, boolean compact)	Telling journal a checkpoint is finished.
(package private) void	forceLedger(long ledgerId, BookkeeperInternalCallbacks.WriteCallback cb, java.lang.Object ctx)
Journal.BufferedChannelBuilder	getBufferedChannelBuilder()
java.io.File	getJournalDirectory()
int	getJournalQueueLength()	Get the length of journal entries queue.
(package private) JournalStats	getJournalStats()
Journal.LastLogMark	getLastLogMark()
(package private) long	getMemoryUsage()
void	interruptThread()
void	joinThread()	Wait for the Journal thread to exit.
static java.util.List<java.lang.Long>	listJournalIds(java.io.File journalDir, Journal.JournalIdFilter filter)	List all journal ids by a specified journal id filer.
void	logAddEntry(long ledgerId, long entryId, io.netty.buffer.ByteBuf entry, boolean ackBeforeSync, BookkeeperInternalCallbacks.WriteCallback cb, java.lang.Object ctx)
void	logAddEntry(io.netty.buffer.ByteBuf entry, boolean ackBeforeSync, BookkeeperInternalCallbacks.WriteCallback cb, java.lang.Object ctx)	record an add entry operation in journal.
CheckpointSource.Checkpoint	newCheckpoint()	Application tried to schedule a checkpoint.
(package private) static Journal	newJournal(int journalIndex, java.io.File journalDirectory, ServerConfiguration conf, LedgerDirsManager ledgerDirsManager, StatsLogger statsLogger, io.netty.buffer.ByteBufAllocator allocator, JournalAliveListener journalAliveListener)
(package private) JournalChannel	newLogFile(long logId, java.lang.Long replaceLogId)
void	run()	A thread used for persisting journal entries to journal files.
long	scanJournal(long journalId, long journalPos, Journal.JournalScanner scanner, boolean skipInvalidRecord)	Scan the journal.
void	setForceWriteRequests(BatchedBlockingQueue<Journal.ForceWriteRequest> forceWriteRequests)
(package private) void	setLastLogMark(java.lang.Long id, long scanOffset)	Update lastLogMark of the journal Indicates that the file has been processed.
(package private) void	setMemoryLimitController(MemoryLimitController memoryLimitController)
void	shutdown()	Shuts down the journal.
void	start()
(package private) static void	writePaddingBytes(JournalChannel jc, io.netty.buffer.ByteBuf paddingBuffer, int journalAlignSize)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PADDING_MASK

static final int PADDING_MASK

See Also:

Constant Field Values

MB

static final long MB

See Also:

Constant Field Values

KB

static final int KB

See Also:

Constant Field Values

maxJournalSize

final long maxJournalSize

journalPreAllocSize

final long journalPreAllocSize

journalWriteBufferSize

final int journalWriteBufferSize

maxBackupJournals

final int maxBackupJournals

journalDirectory

final java.io.File journalDirectory

conf

final ServerConfiguration conf

forceWriteThread

final org.apache.bookkeeper.bookie.Journal.ForceWriteThread forceWriteThread

fileChannelProvider

final FileChannelProvider fileChannelProvider

queue

final BatchedBlockingQueue<Journal.QueueEntry> queue

forceWriteRequests

BatchedBlockingQueue<Journal.ForceWriteRequest> forceWriteRequests

running

volatile boolean running

Constructor Detail

Journal

public Journal(int journalIndex,
               java.io.File journalDirectory,
               ServerConfiguration conf,
               LedgerDirsManager ledgerDirsManager)

Journal

public Journal(int journalIndex,
               java.io.File journalDirectory,
               ServerConfiguration conf,
               LedgerDirsManager ledgerDirsManager,
               StatsLogger statsLogger,
               io.netty.buffer.ByteBufAllocator allocator)

Journal

public Journal(int journalIndex,
               java.io.File journalDirectory,
               ServerConfiguration conf,
               LedgerDirsManager ledgerDirsManager,
               StatsLogger statsLogger,
               io.netty.buffer.ByteBufAllocator allocator,
               JournalAliveListener journalAliveListener)

Method Detail

listJournalIds

public static java.util.List<java.lang.Long> listJournalIds(java.io.File journalDir,
                                                            Journal.JournalIdFilter filter)

List all journal ids by a specified journal id filer.

Parameters:

journalDir - journal dir

filter - journal id filter

Returns:

list of filtered ids

writePaddingBytes

static void writePaddingBytes(JournalChannel jc,
                              io.netty.buffer.ByteBuf paddingBuffer,
                              int journalAlignSize)
                       throws java.io.IOException

Throws:

java.io.IOException

newJournal

static Journal newJournal(int journalIndex,
                          java.io.File journalDirectory,
                          ServerConfiguration conf,
                          LedgerDirsManager ledgerDirsManager,
                          StatsLogger statsLogger,
                          io.netty.buffer.ByteBufAllocator allocator,
                          JournalAliveListener journalAliveListener)

getJournalStats

JournalStats getJournalStats()

getJournalDirectory

public java.io.File getJournalDirectory()

getLastLogMark

public Journal.LastLogMark getLastLogMark()

setLastLogMark

void setLastLogMark(java.lang.Long id,
                    long scanOffset)

Update lastLogMark of the journal Indicates that the file has been processed.

Parameters:

id -

scanOffset -

newCheckpoint

public CheckpointSource.Checkpoint newCheckpoint()

Application tried to schedule a checkpoint. After all the txns added before checkpoint are persisted, a checkpoint will be returned to application. Application could use checkpoint to do its logic.

Specified by:

newCheckpoint in interface CheckpointSource

Returns:

checkpoint.

checkpointComplete

public void checkpointComplete(CheckpointSource.Checkpoint checkpoint,
                               boolean compact)
                        throws java.io.IOException

Telling journal a checkpoint is finished.

Specified by:

checkpointComplete in interface CheckpointSource

Parameters:

checkpoint - The checkpoint that has been completed

compact - Flag to compact old checkpoints.

Throws:

java.io.IOException

scanJournal

public long scanJournal(long journalId,
                        long journalPos,
                        Journal.JournalScanner scanner,
                        boolean skipInvalidRecord)
                 throws java.io.IOException

Scan the journal.

Parameters:

journalId - Journal Log Id

journalPos - Offset to start scanning

scanner - Scanner to handle entries

skipInvalidRecord - when invalid record,should we skip it or not

Returns:

scanOffset - represents the byte till which journal was read

Throws:

java.io.IOException

logAddEntry

public void logAddEntry(io.netty.buffer.ByteBuf entry,
                        boolean ackBeforeSync,
                        BookkeeperInternalCallbacks.WriteCallback cb,
                        java.lang.Object ctx)
                 throws java.lang.InterruptedException

record an add entry operation in journal.

Throws:

java.lang.InterruptedException

logAddEntry

public void logAddEntry(long ledgerId,
                        long entryId,
                        io.netty.buffer.ByteBuf entry,
                        boolean ackBeforeSync,
                        BookkeeperInternalCallbacks.WriteCallback cb,
                        java.lang.Object ctx)
                 throws java.lang.InterruptedException

Throws:

java.lang.InterruptedException

forceLedger

void forceLedger(long ledgerId,
                 BookkeeperInternalCallbacks.WriteCallback cb,
                 java.lang.Object ctx)

getJournalQueueLength

public int getJournalQueueLength()

Get the length of journal entries queue.

Returns:

length of journal entry queue.

newLogFile

JournalChannel newLogFile(long logId,
                          java.lang.Long replaceLogId)
                   throws java.io.IOException

Throws:

java.io.IOException

run

public void run()

A thread used for persisting journal entries to journal files.

Besides persisting journal entries, it also takes responsibility of rolling journal files when a journal file reaches journal file size limitation.

During journal rolling, it first closes the writing journal, generates new journal file using current timestamp, and continue persistence logic. Those journals will be garbage collected in SyncThread.

See Also:

SyncThread

getBufferedChannelBuilder

public Journal.BufferedChannelBuilder getBufferedChannelBuilder()

shutdown

public void shutdown()

Shuts down the journal.

joinThread

public void joinThread()
                throws java.lang.InterruptedException

Wait for the Journal thread to exit. This is method is needed in order to mock the journal, we can't mock final method of java.lang.Thread class

Throws:

java.lang.InterruptedException

interruptThread

public void interruptThread()

start

public void start()

getMemoryUsage

long getMemoryUsage()

setMemoryLimitController

void setMemoryLimitController(MemoryLimitController memoryLimitController)

setForceWriteRequests

public void setForceWriteRequests(BatchedBlockingQueue<Journal.ForceWriteRequest> forceWriteRequests)