Package org.apache.bookkeeper.meta

Class ZkLedgerUnderreplicationManager

java.lang.Object

org.apache.bookkeeper.meta.ZkLedgerUnderreplicationManager

All Implemented Interfaces:

AutoCloseable, LedgerUnderreplicationManager

public class ZkLedgerUnderreplicationManager
extends Object
implements LedgerUnderreplicationManager

ZooKeeper implementation of underreplication manager. This is implemented in a hierarchical fashion, so it'll work with FlatLedgerManagerFactory and HierarchicalLedgerManagerFactory.

Layout is: /root/underreplication/ LAYOUT ledgers/(hierarchicalpath)/urL(ledgerId) locks/(ledgerId)

The hierarchical path is created by splitting the ledger into 4 2byte segments which are represented in hexadecimal. e.g. For ledger id 0xcafebeef0000feed, the path is cafe/beef/0000/feed/

Field Summary

Fields

Modifier and Type	Field	Description
(package private) static final String	LAYOUT
(package private) static final int	LAYOUT_VERSION
(package private) static final org.slf4j.Logger	LOG

Constructor Summary

Constructors

Constructor	Description
ZkLedgerUnderreplicationManager(AbstractConfiguration conf, org.apache.zookeeper.ZooKeeper zkc)

Method Summary

Modifier and Type	Method	Description
void	acquireUnderreplicatedLedger(long ledgerId)
static String	acquireUnderreplicatedLedgerLock(org.apache.zookeeper.ZooKeeper zkc, String zkLedgersRootPath, long ledgerId, List<org.apache.zookeeper.data.ACL> zkAcls)	Acquire the underreplicated ledger lock.
void	close()	Release all resources held by the ledger underreplication manager.
void	disableLedgerReplication()	Stop ledger replication.
void	enableLedgerReplication()	Resuming ledger replication.
static String	getBasePath(String rootPath)
long	getCheckAllLedgersCTime()	Getter for the CheckAllLedgers last executed ctime.
long	getLedgerToRereplicate()	Acquire a underreplicated ledger for rereplication.
UnderreplicatedLedger	getLedgerUnreplicationInfo(long ledgerId)	Get the UnderreplicatedLedger info if this ledger is marked underreplicated otherwise it returns null.
static byte[]	getLockData()
int	getLostBookieRecoveryDelay()	Getter for the lostBookieRecoveryDelay.
static String	getParentZnodePath(String base, long ledgerId)
long	getPlacementPolicyCheckCTime()	Getter for the PlacementPolicyCheck last executed ctime.
long	getReplicasCheckCTime()	Getter for the ReplicasCheck last executed ctime.
String	getReplicationWorkerIdRereplicatingLedger(long ledgerId)	If a replicationworker has acquired lock on an underreplicated ledger, then getReplicationWorkerIdRereplicatingLedger should return ReplicationWorkerId (BookieId) of the ReplicationWorker that is holding lock.
static String	getUrLedgerLockZnode(String base, long ledgerId)
static String	getUrLedgerZnode(String base, long ledgerId)
static String	getUrLockPath(String rootPath)
boolean	initializeLostBookieRecoveryDelay(int lostBookieRecoveryDelay)	Creates the zNode for lostBookieRecoveryDelay with the specified value and returns true.
boolean	isLedgerBeingReplicated(long ledgerId)	Check whether the ledger is being replicated by any bookie.
boolean	isLedgerReplicationEnabled()	Check whether the ledger replication is enabled or not.
Iterator<UnderreplicatedLedger>	listLedgersToRereplicate(Predicate<List<String>> predicate)	Get a list of all the underreplicated ledgers which have been marked for rereplication, filtered by the predicate on the replicas list.
void	markLedgerReplicated(long ledgerId)	Mark a ledger as fully replicated.
CompletableFuture<Void>	markLedgerUnderreplicatedAsync(long ledgerId, Collection<String> missingReplicas)	Mark a ledger as underreplicated with missing bookies.
void	notifyLedgerReplicationEnabled(BookkeeperInternalCallbacks.GenericCallback<Void> cb)	Receive notification asynchronously when the ledger replication process is enabled.
void	notifyLostBookieRecoveryDelayChanged(BookkeeperInternalCallbacks.GenericCallback<Void> cb)	Receive notification asynchronously when the lostBookieRecoveryDelay value is Changed.
void	notifyUnderReplicationLedgerChanged(BookkeeperInternalCallbacks.GenericCallback<Void> cb)	Receive notification asynchronously when the num of under-replicated ledgers Changed.
long	pollLedgerToRereplicate()	Poll for a underreplicated ledger to rereplicate.
void	releaseUnderreplicatedLedger(long ledgerId)	Release a previously acquired ledger.
void	setCheckAllLedgersCTime(long checkAllLedgersCTime)	Setter for the CheckAllLedgers last executed ctime.
void	setLostBookieRecoveryDelay(int lostBookieRecoveryDelay)	Setter for the lostBookieRecoveryDelay znode.
void	setPlacementPolicyCheckCTime(long placementPolicyCheckCTime)	Setter for the PlacementPolicyCheck last executed ctime.
void	setReplicasCheckCTime(long replicasCheckCTime)	Setter for the ReplicasCheck last executed ctime.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.bookkeeper.meta.LedgerUnderreplicationManager

markLedgerUnderreplicated

Field Details

LOG

static final org.slf4j.Logger LOG

LAYOUT

static final String LAYOUT

See Also:

• Constant Field Values

LAYOUT_VERSION

static final int LAYOUT_VERSION

See Also:

• Constant Field Values

Constructor Details

ZkLedgerUnderreplicationManager

public ZkLedgerUnderreplicationManager(AbstractConfiguration conf,
 org.apache.zookeeper.ZooKeeper zkc)
                                throws ReplicationException.UnavailableException,
InterruptedException,
ReplicationException.CompatibilityException

Throws:

ReplicationException.UnavailableException

InterruptedException

ReplicationException.CompatibilityException

Method Details

getBasePath

public static String getBasePath(String rootPath)

getUrLockPath

public static String getUrLockPath(String rootPath)

getLockData

public static byte[] getLockData()

getParentZnodePath

public static String getParentZnodePath(String base,
 long ledgerId)

getUrLedgerZnode

public static String getUrLedgerZnode(String base,
 long ledgerId)

getUrLedgerLockZnode

public static String getUrLedgerLockZnode(String base,
 long ledgerId)

getLedgerUnreplicationInfo

public UnderreplicatedLedger getLedgerUnreplicationInfo(long ledgerId)
                                                 throws ReplicationException.UnavailableException

Description copied from interface: LedgerUnderreplicationManager

Get the UnderreplicatedLedger info if this ledger is marked underreplicated otherwise it returns null.

Specified by:

getLedgerUnreplicationInfo in interface LedgerUnderreplicationManager

Parameters:

ledgerId - ledger id

Returns:

the UnderreplicatedLedger info instance if this ledger is marked underreplicated otherwise it returns null.

Throws:

ReplicationException.UnavailableException

markLedgerUnderreplicatedAsync

public CompletableFuture<Void> markLedgerUnderreplicatedAsync(long ledgerId,
 Collection<String> missingReplicas)

Description copied from interface: LedgerUnderreplicationManager

Mark a ledger as underreplicated with missing bookies. The replication should then check which fragments are underreplicated and rereplicate them.

Specified by:

markLedgerUnderreplicatedAsync in interface LedgerUnderreplicationManager

Parameters:

ledgerId - ledger id

missingReplicas - missing replicas

Returns:

a future presents the mark result.

markLedgerReplicated

public void markLedgerReplicated(long ledgerId)
                          throws ReplicationException.UnavailableException

Description copied from interface: LedgerUnderreplicationManager

Mark a ledger as fully replicated. If the ledger is not already marked as underreplicated, this is a noop.

Specified by:

markLedgerReplicated in interface LedgerUnderreplicationManager

Throws:

ReplicationException.UnavailableException

listLedgersToRereplicate

public Iterator<UnderreplicatedLedger> listLedgersToRereplicate(Predicate<List<String>> predicate)

Get a list of all the underreplicated ledgers which have been marked for rereplication, filtered by the predicate on the replicas list.

Replicas list of an underreplicated ledger is the list of the bookies which are part of the ensemble of this ledger and are currently unavailable/down.

Specified by:

listLedgersToRereplicate in interface LedgerUnderreplicationManager

Parameters:

predicate - filter to use while listing under replicated ledgers. 'null' if filtering is not required.

Returns:

an iterator which returns underreplicated ledgers.

pollLedgerToRereplicate

public long pollLedgerToRereplicate()
                             throws ReplicationException.UnavailableException

Description copied from interface: LedgerUnderreplicationManager

Poll for a underreplicated ledger to rereplicate.

Specified by:

pollLedgerToRereplicate in interface LedgerUnderreplicationManager

Returns:

the ledgerId, or -1 if none are available

Throws:

ReplicationException.UnavailableException

See Also:

• LedgerUnderreplicationManager.getLedgerToRereplicate()

getLedgerToRereplicate

public long getLedgerToRereplicate()
                            throws ReplicationException.UnavailableException

Description copied from interface: LedgerUnderreplicationManager

Acquire a underreplicated ledger for rereplication. The ledger should be locked, so that no other agent will receive the ledger from this call. The ledger should remain locked until either #markLedgerComplete or #releaseLedger are called. This call is blocking, so will not return until a ledger is available for rereplication.

Specified by:

getLedgerToRereplicate in interface LedgerUnderreplicationManager

Throws:

ReplicationException.UnavailableException

releaseUnderreplicatedLedger

public void releaseUnderreplicatedLedger(long ledgerId)
                                  throws ReplicationException.UnavailableException

Description copied from interface: LedgerUnderreplicationManager

Release a previously acquired ledger. This allows others to acquire the ledger.

Specified by:

releaseUnderreplicatedLedger in interface LedgerUnderreplicationManager

Throws:

ReplicationException.UnavailableException

close

public void close()
           throws ReplicationException.UnavailableException

Description copied from interface: LedgerUnderreplicationManager

Release all resources held by the ledger underreplication manager.

Specified by:

close in interface AutoCloseable

Specified by:

close in interface LedgerUnderreplicationManager

Throws:

ReplicationException.UnavailableException

disableLedgerReplication

public void disableLedgerReplication()
                              throws ReplicationException.UnavailableException

Description copied from interface: LedgerUnderreplicationManager

Stop ledger replication. Currently running ledger rereplication tasks will be continued and will be stopped from next task. This will block ledger replication #Auditor and LedgerUnderreplicationManager.getLedgerToRereplicate() tasks.

Specified by:

disableLedgerReplication in interface LedgerUnderreplicationManager

Throws:

ReplicationException.UnavailableException

enableLedgerReplication

public void enableLedgerReplication()
                             throws ReplicationException.UnavailableException

Description copied from interface: LedgerUnderreplicationManager

Resuming ledger replication. This will allow ledger replication #Auditor and LedgerUnderreplicationManager.getLedgerToRereplicate() tasks to continue.

Specified by:

enableLedgerReplication in interface LedgerUnderreplicationManager

Throws:

ReplicationException.UnavailableException

isLedgerReplicationEnabled

public boolean isLedgerReplicationEnabled()
                                   throws ReplicationException.UnavailableException

Description copied from interface: LedgerUnderreplicationManager

Check whether the ledger replication is enabled or not. This will return true if the ledger replication is enabled, otherwise return false.

Specified by:

isLedgerReplicationEnabled in interface LedgerUnderreplicationManager

Returns:

- return true if it is enabled otherwise return false

Throws:

ReplicationException.UnavailableException

notifyLedgerReplicationEnabled

public void notifyLedgerReplicationEnabled(BookkeeperInternalCallbacks.GenericCallback<Void> cb)
                                    throws ReplicationException.UnavailableException

Description copied from interface: LedgerUnderreplicationManager

Receive notification asynchronously when the ledger replication process is enabled.

Specified by:

notifyLedgerReplicationEnabled in interface LedgerUnderreplicationManager

Parameters:

cb - - callback implementation to receive the notification

Throws:

ReplicationException.UnavailableException

isLedgerBeingReplicated

public boolean isLedgerBeingReplicated(long ledgerId)
                                throws ReplicationException

Check whether the ledger is being replicated by any bookie.

Specified by:

isLedgerBeingReplicated in interface LedgerUnderreplicationManager

Throws:

ReplicationException

acquireUnderreplicatedLedgerLock

public static String acquireUnderreplicatedLedgerLock(org.apache.zookeeper.ZooKeeper zkc,
 String zkLedgersRootPath,
 long ledgerId,
 List<org.apache.zookeeper.data.ACL> zkAcls)
                                               throws ReplicationException.UnavailableException,
InterruptedException

Acquire the underreplicated ledger lock.

Throws:

ReplicationException.UnavailableException

InterruptedException

acquireUnderreplicatedLedger

public void acquireUnderreplicatedLedger(long ledgerId)
                                  throws ReplicationException

Specified by:

acquireUnderreplicatedLedger in interface LedgerUnderreplicationManager

Throws:

ReplicationException

initializeLostBookieRecoveryDelay

public boolean initializeLostBookieRecoveryDelay(int lostBookieRecoveryDelay)
                                          throws ReplicationException.UnavailableException

Description copied from interface: LedgerUnderreplicationManager

Creates the zNode for lostBookieRecoveryDelay with the specified value and returns true. If the node is already existing, then it returns false.

Specified by:

initializeLostBookieRecoveryDelay in interface LedgerUnderreplicationManager

Returns:

true if it succeeds in creating zNode for lostBookieRecoveryDelay, false if it is already existing

Throws:

ReplicationException.UnavailableException

setLostBookieRecoveryDelay

public void setLostBookieRecoveryDelay(int lostBookieRecoveryDelay)
                                throws ReplicationException.UnavailableException

Description copied from interface: LedgerUnderreplicationManager

Setter for the lostBookieRecoveryDelay znode.

Specified by:

setLostBookieRecoveryDelay in interface LedgerUnderreplicationManager

Throws:

ReplicationException.UnavailableException

getLostBookieRecoveryDelay

public int getLostBookieRecoveryDelay()
                               throws ReplicationException.UnavailableException

Description copied from interface: LedgerUnderreplicationManager

Getter for the lostBookieRecoveryDelay.

Specified by:

getLostBookieRecoveryDelay in interface LedgerUnderreplicationManager

Returns:

the int value of lostBookieRecoveryDelay

Throws:

ReplicationException.UnavailableException

notifyUnderReplicationLedgerChanged

public void notifyUnderReplicationLedgerChanged(BookkeeperInternalCallbacks.GenericCallback<Void> cb)
                                         throws ReplicationException.UnavailableException

Description copied from interface: LedgerUnderreplicationManager

Receive notification asynchronously when the num of under-replicated ledgers Changed.

Specified by:

notifyUnderReplicationLedgerChanged in interface LedgerUnderreplicationManager

Throws:

ReplicationException.UnavailableException

notifyLostBookieRecoveryDelayChanged

public void notifyLostBookieRecoveryDelayChanged(BookkeeperInternalCallbacks.GenericCallback<Void> cb)
                                          throws ReplicationException.UnavailableException

Description copied from interface: LedgerUnderreplicationManager

Receive notification asynchronously when the lostBookieRecoveryDelay value is Changed.

Specified by:

notifyLostBookieRecoveryDelayChanged in interface LedgerUnderreplicationManager

Throws:

ReplicationException.UnavailableException

getReplicationWorkerIdRereplicatingLedger

public String getReplicationWorkerIdRereplicatingLedger(long ledgerId)
                                                 throws ReplicationException.UnavailableException

Description copied from interface: LedgerUnderreplicationManager

If a replicationworker has acquired lock on an underreplicated ledger, then getReplicationWorkerIdRereplicatingLedger should return ReplicationWorkerId (BookieId) of the ReplicationWorker that is holding lock. If lock for the underreplicated ledger is not yet acquired or if it is released then it is supposed to return null.

Specified by:

getReplicationWorkerIdRereplicatingLedger in interface LedgerUnderreplicationManager

Returns:

Throws:

ReplicationException.UnavailableException

setCheckAllLedgersCTime

public void setCheckAllLedgersCTime(long checkAllLedgersCTime)
                             throws ReplicationException.UnavailableException

Description copied from interface: LedgerUnderreplicationManager

Setter for the CheckAllLedgers last executed ctime.

Specified by:

setCheckAllLedgersCTime in interface LedgerUnderreplicationManager

Throws:

ReplicationException.UnavailableException

getCheckAllLedgersCTime

public long getCheckAllLedgersCTime()
                             throws ReplicationException.UnavailableException

Description copied from interface: LedgerUnderreplicationManager

Getter for the CheckAllLedgers last executed ctime.

Specified by:

getCheckAllLedgersCTime in interface LedgerUnderreplicationManager

Returns:

the long value of checkAllLedgersCTime

Throws:

ReplicationException.UnavailableException

setPlacementPolicyCheckCTime

public void setPlacementPolicyCheckCTime(long placementPolicyCheckCTime)
                                  throws ReplicationException.UnavailableException

Description copied from interface: LedgerUnderreplicationManager

Setter for the PlacementPolicyCheck last executed ctime.

Specified by:

setPlacementPolicyCheckCTime in interface LedgerUnderreplicationManager

Throws:

ReplicationException.UnavailableException

getPlacementPolicyCheckCTime

public long getPlacementPolicyCheckCTime()
                                  throws ReplicationException.UnavailableException

Description copied from interface: LedgerUnderreplicationManager

Getter for the PlacementPolicyCheck last executed ctime.

Specified by:

getPlacementPolicyCheckCTime in interface LedgerUnderreplicationManager

Returns:

the long value of placementPolicyCheckCTime

Throws:

ReplicationException.UnavailableException

setReplicasCheckCTime

public void setReplicasCheckCTime(long replicasCheckCTime)
                           throws ReplicationException.UnavailableException

Description copied from interface: LedgerUnderreplicationManager

Setter for the ReplicasCheck last executed ctime.

Specified by:

setReplicasCheckCTime in interface LedgerUnderreplicationManager

Throws:

ReplicationException.UnavailableException

getReplicasCheckCTime

public long getReplicasCheckCTime()
                           throws ReplicationException.UnavailableException

Description copied from interface: LedgerUnderreplicationManager

Getter for the ReplicasCheck last executed ctime.

Specified by:

getReplicasCheckCTime in interface LedgerUnderreplicationManager

Returns:

the long value of replicasCheckCTime

Throws:

ReplicationException.UnavailableException