Package org.apache.bookkeeper.bookie

Class GarbageCollectorThread

java.lang.Object

org.apache.bookkeeper.bookie.GarbageCollectorThread

All Implemented Interfaces:

java.lang.Runnable

public class GarbageCollectorThread
extends java.lang.Object
implements java.lang.Runnable

This is the garbage collector thread that runs in the background to remove any entry log files that no longer contains any active ledger.

Field Summary

Fields

Modifier and Type	Field	Description
(package private) java.util.concurrent.atomic.AtomicBoolean	compacting
(package private) AbstractLogCompactor	compactor
(package private) ServerConfiguration	conf
(package private) boolean	enableMajorCompaction
(package private) boolean	enableMinorCompaction
(package private) EntryLogger	entryLogger
(package private) java.util.concurrent.atomic.AtomicBoolean	forceGarbageCollection
(package private) GarbageCollector.GarbageCleaner	garbageCleaner
(package private) ScanAndCompareGarbageCollector	garbageCollector
(package private) long	gcWaitTime
(package private) boolean	isForceGCAllowWhenNoSpace
(package private) boolean	isForceMajorCompactionAllow
(package private) boolean	isForceMinorCompactionAllow
(package private) long	lastMajorCompactionTime
(package private) long	lastMinorCompactionTime
(package private) LedgerDirsManager	ledgerDirsManager
(package private) CompactableLedgerStorage	ledgerStorage
(package private) java.util.concurrent.atomic.AtomicBoolean	majorCompacting
(package private) long	majorCompactionInterval
(package private) long	majorCompactionMaxTimeMillis
(package private) double	majorCompactionThreshold
(package private) java.util.concurrent.atomic.AtomicBoolean	minorCompacting
(package private) long	minorCompactionInterval
(package private) long	minorCompactionMaxTimeMillis
(package private) double	minorCompactionThreshold
(package private) boolean	running
(package private) java.util.concurrent.Future<?>	scheduledFuture
(package private) java.util.concurrent.atomic.AtomicBoolean	suspendMajorCompaction
(package private) java.util.concurrent.atomic.AtomicBoolean	suspendMinorCompaction
(package private) AbstractLogCompactor.Throttler	throttler

Constructor Summary

Constructors

Constructor	Description
GarbageCollectorThread(ServerConfiguration conf, LedgerManager ledgerManager, LedgerDirsManager ledgerDirsManager, CompactableLedgerStorage ledgerStorage, EntryLogger entryLogger, StatsLogger statsLogger)	Create a garbage collector thread.
GarbageCollectorThread(ServerConfiguration conf, LedgerManager ledgerManager, LedgerDirsManager ledgerDirsManager, CompactableLedgerStorage ledgerStorage, EntryLogger entryLogger, StatsLogger statsLogger, java.util.concurrent.ScheduledExecutorService gcExecutor)	Create a garbage collector thread.

Method Summary

Modifier and Type	Method	Description
(package private) int	calculateUsageIndex(int numBuckets, double usage)	Calculate the index for the batch based on the usage between 0 and 1.
protected void	compactEntryLog(EntryLogMetadata entryLogMeta)	Compact an entry log.
void	disableForceGC()
(package private) void	doCompactEntryLogs(double threshold, long maxTimeMillis)	Compact entry logs if necessary.
void	enableForceGC()
void	enableForceGC(boolean forceMajor, boolean forceMinor)
protected void	extractMetaFromEntryLogs()	Method to read in all of the entry logs (those that we haven't done so yet), and find the set of ledger ID's that make up each entry log file.
(package private) EntryLogMetadataMap	getEntryLogMetaMap()
GarbageCollectionStatus	getGarbageCollectionStatus()
(package private) CompactableLedgerStorage	getLedgerStorage()
long	getModInitialDelay()	when number of ledger's Dir are more than 1,the same of GarbageCollectorThread will do the same thing, Especially 1) deleting ledger, then SyncThread will be timed to do rocksDB compact 2) compact: entry, cost cpu.
boolean	isForceGCAllowWhenNoSpace()
boolean	isInForceGC()
boolean	isMajorGcSuspend()
boolean	isMinorGcSuspend()
(package private) static java.util.concurrent.ScheduledExecutorService	newExecutor()
protected boolean	removeEntryLog(long entryLogId)	Remove entry log.
void	resumeMajorGC()
void	resumeMinorGC()
void	run()
void	runWithFlags(boolean force, boolean suspendMajor, boolean suspendMinor)
void	shutdown()	Shutdown the garbage collector thread.
void	start()
void	suspendMajorGC()
void	suspendMinorGC()
(package private) java.util.concurrent.Future<?>	triggerGC()
(package private) java.util.concurrent.Future<?>	triggerGC(boolean force, boolean suspendMajor, boolean suspendMinor)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

scheduledFuture

java.util.concurrent.Future<?> scheduledFuture

gcWaitTime

final long gcWaitTime

isForceMinorCompactionAllow

boolean isForceMinorCompactionAllow

enableMinorCompaction

boolean enableMinorCompaction

minorCompactionThreshold

final double minorCompactionThreshold

minorCompactionInterval

final long minorCompactionInterval

minorCompactionMaxTimeMillis

final long minorCompactionMaxTimeMillis

lastMinorCompactionTime

long lastMinorCompactionTime

isForceMajorCompactionAllow

boolean isForceMajorCompactionAllow

enableMajorCompaction

boolean enableMajorCompaction

majorCompactionThreshold

final double majorCompactionThreshold

majorCompactionInterval

final long majorCompactionInterval

majorCompactionMaxTimeMillis

long majorCompactionMaxTimeMillis

lastMajorCompactionTime

long lastMajorCompactionTime

isForceGCAllowWhenNoSpace

final boolean isForceGCAllowWhenNoSpace

entryLogger

final EntryLogger entryLogger

compactor

AbstractLogCompactor compactor

ledgerStorage

final CompactableLedgerStorage ledgerStorage

compacting

final java.util.concurrent.atomic.AtomicBoolean compacting

minorCompacting

final java.util.concurrent.atomic.AtomicBoolean minorCompacting

majorCompacting

final java.util.concurrent.atomic.AtomicBoolean majorCompacting

running

volatile boolean running

forceGarbageCollection

final java.util.concurrent.atomic.AtomicBoolean forceGarbageCollection

suspendMajorCompaction

final java.util.concurrent.atomic.AtomicBoolean suspendMajorCompaction

suspendMinorCompaction

final java.util.concurrent.atomic.AtomicBoolean suspendMinorCompaction

garbageCollector

final ScanAndCompareGarbageCollector garbageCollector

garbageCleaner

final GarbageCollector.GarbageCleaner garbageCleaner

conf

final ServerConfiguration conf

ledgerDirsManager

final LedgerDirsManager ledgerDirsManager

throttler

final AbstractLogCompactor.Throttler throttler

Constructor Detail

GarbageCollectorThread

public GarbageCollectorThread(ServerConfiguration conf,
                              LedgerManager ledgerManager,
                              LedgerDirsManager ledgerDirsManager,
                              CompactableLedgerStorage ledgerStorage,
                              EntryLogger entryLogger,
                              StatsLogger statsLogger)
                       throws java.io.IOException

Create a garbage collector thread.

Parameters:

conf - Server Configuration Object.

Throws:

java.io.IOException

GarbageCollectorThread

public GarbageCollectorThread(ServerConfiguration conf,
                              LedgerManager ledgerManager,
                              LedgerDirsManager ledgerDirsManager,
                              CompactableLedgerStorage ledgerStorage,
                              EntryLogger entryLogger,
                              StatsLogger statsLogger,
                              java.util.concurrent.ScheduledExecutorService gcExecutor)
                       throws java.io.IOException

Create a garbage collector thread.

Parameters:

conf - Server Configuration Object.

Throws:

java.io.IOException

Method Detail

newExecutor

static java.util.concurrent.ScheduledExecutorService newExecutor()

enableForceGC

public void enableForceGC()

enableForceGC

public void enableForceGC(boolean forceMajor,
                          boolean forceMinor)

disableForceGC

public void disableForceGC()

triggerGC

java.util.concurrent.Future<?> triggerGC(boolean force,
                                         boolean suspendMajor,
                                         boolean suspendMinor)

triggerGC

java.util.concurrent.Future<?> triggerGC()

isInForceGC

public boolean isInForceGC()

isMajorGcSuspend

public boolean isMajorGcSuspend()

isMinorGcSuspend

public boolean isMinorGcSuspend()

suspendMajorGC

public void suspendMajorGC()

resumeMajorGC

public void resumeMajorGC()

suspendMinorGC

public void suspendMinorGC()

resumeMinorGC

public void resumeMinorGC()

start

public void start()

getModInitialDelay

public long getModInitialDelay()

when number of ledger's Dir are more than 1,the same of GarbageCollectorThread will do the same thing, Especially 1) deleting ledger, then SyncThread will be timed to do rocksDB compact 2) compact: entry, cost cpu. then get Mod initial Delay time to simply avoid GarbageCollectorThread working at the same time

run

public void run()

Specified by:

run in interface java.lang.Runnable

runWithFlags

public void runWithFlags(boolean force,
                         boolean suspendMajor,
                         boolean suspendMinor)

doCompactEntryLogs

void doCompactEntryLogs(double threshold,
                        long maxTimeMillis)
                 throws BookieException.EntryLogMetadataMapException

Compact entry logs if necessary.

Compaction will be executed from low unused space to high unused space. Those entry log files whose remaining size percentage is higher than threshold would not be compacted.

Throws:

BookieException.EntryLogMetadataMapException

calculateUsageIndex

int calculateUsageIndex(int numBuckets,
                        double usage)

Calculate the index for the batch based on the usage between 0 and 1.

Parameters:

numBuckets - Number of reporting buckets.

usage - 0.0 - 1.0 value representing the usage of the entry log.

Returns:

index based on the number of buckets The last bucket will have the 1.0 if added.

shutdown

public void shutdown()
              throws java.lang.InterruptedException

Shutdown the garbage collector thread.

Throws:

java.lang.InterruptedException - if there is an exception stopping gc thread.

removeEntryLog

protected boolean removeEntryLog(long entryLogId)
                          throws BookieException.EntryLogMetadataMapException

Remove entry log.

Parameters:

entryLogId - Entry Log File Id

Throws:

BookieException.EntryLogMetadataMapException

compactEntryLog

protected void compactEntryLog(EntryLogMetadata entryLogMeta)

Compact an entry log.

Parameters:

entryLogMeta -

extractMetaFromEntryLogs

protected void extractMetaFromEntryLogs()
                                 throws BookieException.EntryLogMetadataMapException

Method to read in all of the entry logs (those that we haven't done so yet), and find the set of ledger ID's that make up each entry log file.

Throws:

BookieException.EntryLogMetadataMapException

getLedgerStorage

CompactableLedgerStorage getLedgerStorage()

getEntryLogMetaMap

EntryLogMetadataMap getEntryLogMetaMap()

getGarbageCollectionStatus

public GarbageCollectionStatus getGarbageCollectionStatus()

isForceGCAllowWhenNoSpace

public boolean isForceGCAllowWhenNoSpace()