Package org.apache.bookkeeper.client

Class RackawareEnsemblePlacementPolicyImpl

java.lang.Object

org.apache.bookkeeper.client.TopologyAwareEnsemblePlacementPolicy

org.apache.bookkeeper.client.RackawareEnsemblePlacementPolicyImpl

All Implemented Interfaces:

EnsemblePlacementPolicy, ITopologyAwareEnsemblePlacementPolicy<BookieNode>

Direct Known Subclasses:

RackawareEnsemblePlacementPolicy

@StatsDoc(name="bookkeeper_client",
          help="BookKeeper client stats")
public class RackawareEnsemblePlacementPolicyImpl
extends TopologyAwareEnsemblePlacementPolicy

Simple rackware ensemble placement policy.

Make most of the class and methods as protected, so it could be extended to implement other algorithms.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.bookkeeper.client.TopologyAwareEnsemblePlacementPolicy

TopologyAwareEnsemblePlacementPolicy.DefaultResolver, TopologyAwareEnsemblePlacementPolicy.DNSResolverDecorator, TopologyAwareEnsemblePlacementPolicy.EnsembleForReplacementWithNoConstraints, TopologyAwareEnsemblePlacementPolicy.RRTopologyAwareCoverageEnsemble, TopologyAwareEnsemblePlacementPolicy.TruePredicate

Nested classes/interfaces inherited from interface org.apache.bookkeeper.client.EnsemblePlacementPolicy

EnsemblePlacementPolicy.PlacementPolicyAdherence, EnsemblePlacementPolicy.PlacementResult<T>

Nested classes/interfaces inherited from interface org.apache.bookkeeper.client.ITopologyAwareEnsemblePlacementPolicy

ITopologyAwareEnsemblePlacementPolicy.Ensemble<T extends Node>, ITopologyAwareEnsemblePlacementPolicy.Predicate<T extends Node>

Field Summary

Fields

Modifier and Type	Field	Description
protected boolean	enforceDurability
protected boolean	enforceMinNumRacksPerWriteQuorum
protected Counter	failedToResolveNetworkLocationCounter
protected boolean	ignoreLocalNodeInPlacementPolicy
(package private) static int	LOCAL_FAIL_MASK
(package private) static int	LOCAL_MASK
protected BookieNode	localNode
(package private) static org.slf4j.Logger	LOG
(package private) static int	MASK_BITS
(package private) int	maxWeightMultiple
protected int	minNumRacksPerWriteQuorum
protected Gauge<java.lang.Integer>	numWritableBookiesInDefaultRack
(package private) static int	RACKNAME_DISTANCE_FROM_LEAVES
(package private) static int	READ_ONLY_MASK
protected OpStatsLogger	readReorderedCounter
(package private) static int	REMOTE_FAIL_MASK
(package private) static int	REMOTE_MASK
protected boolean	reorderReadsRandom
protected int	reorderThresholdPendingRequests
static java.lang.String	REPP_RANDOM_READ_REORDERING
(package private) static int	SLOW_MASK
protected com.google.common.cache.Cache<BookieId,java.lang.Long>	slowBookies
protected int	stabilizePeriodSeconds
protected StatsLogger	statsLogger
protected io.netty.util.HashedWheelTimer	timer
(package private) static int	UNAVAIL_MASK
protected boolean	useHostnameResolveLocalNodePlacementPolicy

Fields inherited from class org.apache.bookkeeper.client.TopologyAwareEnsemblePlacementPolicy

bookieAddressResolver, bookieInfoMap, bookiesJoinedCounter, bookiesLeftCounter, dnsResolver, historyBookies, isWeighted, knownBookies, readOnlyBookies, REPP_DNS_RESOLVER_CLASS, rwLock, topology, weightedSelection

Constructor Summary

Constructors

Constructor	Description
RackawareEnsemblePlacementPolicyImpl()
RackawareEnsemblePlacementPolicyImpl(boolean enforceDurability)

Method Summary

Modifier and Type	Method	Description
protected java.util.Set<BookieId>	addDefaultRackBookiesIfMinNumRacksIsEnforced(java.util.Set<BookieId> excludeBookies)
boolean	areAckedBookiesAdheringToPlacementPolicy(java.util.Set<BookieId> ackedBookies, int writeQuorumSize, int ackQuorumSize)	Returns true if the bookies that have acknowledged a write adhere to the minimum fault domains as defined in the placement policy in use.
java.lang.String	getDefaultRack()
RackawareEnsemblePlacementPolicyImpl	initialize(ClientConfiguration conf, java.util.Optional<DNSToSwitchMapping> optionalDnsResolver, io.netty.util.HashedWheelTimer timer, FeatureProvider featureProvider, StatsLogger statsLogger, BookieAddressResolver bookieAddressResolver)	Initialize the policy.
protected RackawareEnsemblePlacementPolicyImpl	initialize(DNSToSwitchMapping dnsResolver, io.netty.util.HashedWheelTimer timer, boolean reorderReadsRandom, int stabilizePeriodSeconds, int reorderThresholdPendingRequests, boolean isWeighted, int maxWeightMultiple, int minNumRacksPerWriteQuorum, boolean enforceMinNumRacksPerWriteQuorum, boolean ignoreLocalNodeInPlacementPolicy, boolean useHostnameResolveLocalNodePlacementPolicy, StatsLogger statsLogger, BookieAddressResolver bookieAddressResolver)	Initialize the policy.
protected RackawareEnsemblePlacementPolicyImpl	initialize(DNSToSwitchMapping dnsResolver, io.netty.util.HashedWheelTimer timer, boolean reorderReadsRandom, int stabilizePeriodSeconds, int reorderThresholdPendingRequests, boolean isWeighted, int maxWeightMultiple, int minNumRacksPerWriteQuorum, boolean enforceMinNumRacksPerWriteQuorum, boolean ignoreLocalNodeInPlacementPolicy, StatsLogger statsLogger, BookieAddressResolver bookieAddressResolver)	Initialize the policy.
EnsemblePlacementPolicy.PlacementPolicyAdherence	isEnsembleAdheringToPlacementPolicy(java.util.List<BookieId> ensembleList, int writeQuorumSize, int ackQuorumSize)	returns AdherenceLevel if the Ensemble is strictly/softly/fails adhering to placement policy, like in the case of RackawareEnsemblePlacementPolicy, bookies in the writeset are from 'minNumRacksPerWriteQuorum' number of racks.
EnsemblePlacementPolicy.PlacementResult<java.util.List<BookieId>>	newEnsemble(int ensembleSize, int writeQuorumSize, int ackQuorumSize, java.util.Map<java.lang.String,byte[]> customMetadata, java.util.Set<BookieId> excludeBookies)	Choose numBookies bookies for ensemble.
EnsemblePlacementPolicy.PlacementResult<java.util.List<BookieId>>	newEnsemble(int ensembleSize, int writeQuorumSize, int ackQuorumSize, java.util.Set<BookieId> excludeBookies, ITopologyAwareEnsemblePlacementPolicy.Ensemble<BookieNode> parentEnsemble, ITopologyAwareEnsemblePlacementPolicy.Predicate<BookieNode> parentPredicate)	Create an ensemble with parent ensemble.
protected EnsemblePlacementPolicy.PlacementResult<java.util.List<BookieId>>	newEnsembleInternal(int ensembleSize, int writeQuorumSize, int ackQuorumSize, java.util.Set<BookieId> excludeBookies, ITopologyAwareEnsemblePlacementPolicy.Ensemble<BookieNode> parentEnsemble, ITopologyAwareEnsemblePlacementPolicy.Predicate<BookieNode> parentPredicate)
void	registerSlowBookie(BookieId bookieSocketAddress, long entryId)	Register a bookie as slow so that it is tried after available and read-only bookies.
DistributionSchedule.WriteSet	reorderReadSequence(java.util.List<BookieId> ensemble, BookiesHealthInfo bookiesHealthInfo, DistributionSchedule.WriteSet writeSet)	Reorder the read sequence of a given write quorum writeSet.
(package private) DistributionSchedule.WriteSet	reorderReadSequenceWithRegion(java.util.List<BookieId> ensemble, DistributionSchedule.WriteSet writeSet, java.util.Map<java.lang.Integer,java.lang.String> writeSetWithRegion, BookiesHealthInfo bookiesHealthInfo, boolean regionAware, java.lang.String myRegion, int remoteNodeInReorderSequence)	This function orders the read sequence with a given region.
EnsemblePlacementPolicy.PlacementResult<BookieId>	replaceBookie(int ensembleSize, int writeQuorumSize, int ackQuorumSize, java.util.Map<java.lang.String,byte[]> customMetadata, java.util.List<BookieId> currentEnsemble, BookieId bookieToReplace, java.util.Set<BookieId> excludeBookies)	Choose a new bookie to replace bookieToReplace.
EnsemblePlacementPolicy.PlacementResult<java.util.List<BookieId>>	replaceToAdherePlacementPolicy(int ensembleSize, int writeQuorumSize, int ackQuorumSize, java.util.Set<BookieId> excludeBookies, java.util.List<BookieId> currentEnsemble)	Returns placement result.
BookieNode	selectFromNetworkLocation(java.lang.String networkLoc, java.util.Set<java.lang.String> excludeRacks, java.util.Set<Node> excludeBookies, ITopologyAwareEnsemblePlacementPolicy.Predicate<BookieNode> predicate, ITopologyAwareEnsemblePlacementPolicy.Ensemble<BookieNode> ensemble, boolean fallbackToRandom)	Select a node from networkLoc rack excluding excludeBookies.
BookieNode	selectFromNetworkLocation(java.lang.String networkLoc, java.util.Set<Node> excludeBookies, ITopologyAwareEnsemblePlacementPolicy.Predicate<BookieNode> predicate, ITopologyAwareEnsemblePlacementPolicy.Ensemble<BookieNode> ensemble, boolean fallbackToRandom)	Select a node from a given network location.
BookieNode	selectFromNetworkLocation(java.util.Set<java.lang.String> excludeRacks, java.util.Set<Node> excludeBookies, ITopologyAwareEnsemblePlacementPolicy.Predicate<BookieNode> predicate, ITopologyAwareEnsemblePlacementPolicy.Ensemble<BookieNode> ensemble, boolean fallbackToRandom)	It randomly selects a BookieNode that is not on the excludeRacks set, excluding the nodes in excludeBookies set.
protected java.util.List<BookieNode>	selectRandom(int numBookies, java.util.Set<Node> excludeBookies, ITopologyAwareEnsemblePlacementPolicy.Predicate<BookieNode> predicate, ITopologyAwareEnsemblePlacementPolicy.Ensemble<BookieNode> ensemble)	Choose a random node from whole cluster.
protected BookieNode	selectRandomFromRack(java.lang.String netPath, java.util.Set<Node> excludeBookies, ITopologyAwareEnsemblePlacementPolicy.Predicate<BookieNode> predicate, ITopologyAwareEnsemblePlacementPolicy.Ensemble<BookieNode> ensemble)	Choose random node under a given network path.
protected java.util.List<BookieNode>	selectRandomInternal(java.util.List<BookieNode> bookiesToSelectFrom, int numBookies, java.util.Set<Node> excludeBookies, ITopologyAwareEnsemblePlacementPolicy.Predicate<BookieNode> predicate, ITopologyAwareEnsemblePlacementPolicy.Ensemble<BookieNode> ensemble)
void	uninitalize()	Uninitialize the policy.
RackawareEnsemblePlacementPolicyImpl	withDefaultRack(java.lang.String rack)

Methods inherited from class org.apache.bookkeeper.client.TopologyAwareEnsemblePlacementPolicy

convertBookiesToNodes, convertBookieToNode, createBookieNode, createDummyLocalBookieNode, differBetweenBookies, getNetworkLocations, handleBookiesThatJoined, handleBookiesThatLeft, onBookieRackChange, onClusterChanged, reorderReadLACSequence, resolveNetworkLocation, shuffleWithMask, updateBookieInfo

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.bookkeeper.client.EnsemblePlacementPolicy

getStickyReadBookieIndex

Field Detail

LOG

static final org.slf4j.Logger LOG

maxWeightMultiple

int maxWeightMultiple

minNumRacksPerWriteQuorum

protected int minNumRacksPerWriteQuorum

enforceMinNumRacksPerWriteQuorum

protected boolean enforceMinNumRacksPerWriteQuorum

ignoreLocalNodeInPlacementPolicy

protected boolean ignoreLocalNodeInPlacementPolicy

useHostnameResolveLocalNodePlacementPolicy

protected boolean useHostnameResolveLocalNodePlacementPolicy

REPP_RANDOM_READ_REORDERING

public static final java.lang.String REPP_RANDOM_READ_REORDERING

See Also:

Constant Field Values

RACKNAME_DISTANCE_FROM_LEAVES

static final int RACKNAME_DISTANCE_FROM_LEAVES

See Also:

Constant Field Values

LOCAL_MASK

static final int LOCAL_MASK

See Also:

Constant Field Values

LOCAL_FAIL_MASK

static final int LOCAL_FAIL_MASK

See Also:

Constant Field Values

REMOTE_MASK

static final int REMOTE_MASK

See Also:

Constant Field Values

REMOTE_FAIL_MASK

static final int REMOTE_FAIL_MASK

See Also:

Constant Field Values

READ_ONLY_MASK

static final int READ_ONLY_MASK

See Also:

Constant Field Values

SLOW_MASK

static final int SLOW_MASK

See Also:

Constant Field Values

UNAVAIL_MASK

static final int UNAVAIL_MASK

See Also:

Constant Field Values

MASK_BITS

static final int MASK_BITS

See Also:

Constant Field Values

timer

protected io.netty.util.HashedWheelTimer timer

slowBookies

protected com.google.common.cache.Cache<BookieId,java.lang.Long> slowBookies

localNode

protected BookieNode localNode

reorderReadsRandom

protected boolean reorderReadsRandom

enforceDurability

protected boolean enforceDurability

stabilizePeriodSeconds

protected int stabilizePeriodSeconds

reorderThresholdPendingRequests

protected int reorderThresholdPendingRequests

statsLogger

protected StatsLogger statsLogger

readReorderedCounter

@StatsDoc(name="READ_REQUESTS_REORDERED",
          help="The distribution of number of bookies reordered on each read request")
protected OpStatsLogger readReorderedCounter

failedToResolveNetworkLocationCounter

@StatsDoc(name="FAILED_TO_RESOLVE_NETWORK_LOCATION_TOTAL",
          help="Counter for number of times DNSResolverDecorator failed to resolve Network Location")
protected Counter failedToResolveNetworkLocationCounter

numWritableBookiesInDefaultRack

@StatsDoc(name="NUM_WRITABLE_BOOKIES_IN_DEFAULT_RACK",
          help="Gauge for the number of writable Bookies in default rack")
protected Gauge<java.lang.Integer> numWritableBookiesInDefaultRack

Constructor Detail

RackawareEnsemblePlacementPolicyImpl

RackawareEnsemblePlacementPolicyImpl()

RackawareEnsemblePlacementPolicyImpl

RackawareEnsemblePlacementPolicyImpl(boolean enforceDurability)

Method Detail

initialize

protected RackawareEnsemblePlacementPolicyImpl initialize(DNSToSwitchMapping dnsResolver,
                                                          io.netty.util.HashedWheelTimer timer,
                                                          boolean reorderReadsRandom,
                                                          int stabilizePeriodSeconds,
                                                          int reorderThresholdPendingRequests,
                                                          boolean isWeighted,
                                                          int maxWeightMultiple,
                                                          int minNumRacksPerWriteQuorum,
                                                          boolean enforceMinNumRacksPerWriteQuorum,
                                                          boolean ignoreLocalNodeInPlacementPolicy,
                                                          StatsLogger statsLogger,
                                                          BookieAddressResolver bookieAddressResolver)

Initialize the policy.

Parameters:

dnsResolver -

timer -

reorderReadsRandom -

stabilizePeriodSeconds -

reorderThresholdPendingRequests -

isWeighted -

maxWeightMultiple -

minNumRacksPerWriteQuorum -

enforceMinNumRacksPerWriteQuorum -

ignoreLocalNodeInPlacementPolicy -

statsLogger -

bookieAddressResolver -

Returns:

initialized ensemble placement policy

initialize

protected RackawareEnsemblePlacementPolicyImpl initialize(DNSToSwitchMapping dnsResolver,
                                                          io.netty.util.HashedWheelTimer timer,
                                                          boolean reorderReadsRandom,
                                                          int stabilizePeriodSeconds,
                                                          int reorderThresholdPendingRequests,
                                                          boolean isWeighted,
                                                          int maxWeightMultiple,
                                                          int minNumRacksPerWriteQuorum,
                                                          boolean enforceMinNumRacksPerWriteQuorum,
                                                          boolean ignoreLocalNodeInPlacementPolicy,
                                                          boolean useHostnameResolveLocalNodePlacementPolicy,
                                                          StatsLogger statsLogger,
                                                          BookieAddressResolver bookieAddressResolver)

Initialize the policy.

Parameters:

dnsResolver - the object used to resolve addresses to their network address

Returns:

initialized ensemble placement policy

withDefaultRack

public RackawareEnsemblePlacementPolicyImpl withDefaultRack(java.lang.String rack)

getDefaultRack

public java.lang.String getDefaultRack()

initialize

public RackawareEnsemblePlacementPolicyImpl initialize(ClientConfiguration conf,
                                                       java.util.Optional<DNSToSwitchMapping> optionalDnsResolver,
                                                       io.netty.util.HashedWheelTimer timer,
                                                       FeatureProvider featureProvider,
                                                       StatsLogger statsLogger,
                                                       BookieAddressResolver bookieAddressResolver)

Description copied from interface: EnsemblePlacementPolicy

Initialize the policy.

Parameters:

conf - client configuration

optionalDnsResolver - dns resolver

timer - timer

featureProvider - feature provider

statsLogger - stats logger

uninitalize

public void uninitalize()

Description copied from interface: EnsemblePlacementPolicy

Uninitialize the policy.

addDefaultRackBookiesIfMinNumRacksIsEnforced

protected java.util.Set<BookieId> addDefaultRackBookiesIfMinNumRacksIsEnforced(java.util.Set<BookieId> excludeBookies)

newEnsemble

public EnsemblePlacementPolicy.PlacementResult<java.util.List<BookieId>> newEnsemble(int ensembleSize,
                                                                                     int writeQuorumSize,
                                                                                     int ackQuorumSize,
                                                                                     java.util.Map<java.lang.String,byte[]> customMetadata,
                                                                                     java.util.Set<BookieId> excludeBookies)
                                                                              throws BKException.BKNotEnoughBookiesException

Description copied from interface: EnsemblePlacementPolicy

Choose numBookies bookies for ensemble. If the count is more than the number of available nodes, BKException.BKNotEnoughBookiesException is thrown.

The implementation should respect to the replace settings. The size of the returned bookie list should be equal to the provide ensembleSize.

customMetadata is the same user defined data that user provides when BookKeeper.createLedger(int, int, int, BookKeeper.DigestType, byte[], Map).

If 'enforceMinNumRacksPerWriteQuorum' config is enabled then the bookies belonging to default faultzone (rack) will be excluded while selecting bookies.

Parameters:

ensembleSize - Ensemble Size

writeQuorumSize - Write Quorum Size

ackQuorumSize - the value of ackQuorumSize (added since 4.5)

customMetadata - the value of customMetadata. it is the same user defined metadata that user provides in BookKeeper.createLedger(int, int, int, BookKeeper.DigestType, byte[])

excludeBookies - Bookies that should not be considered as targets.

Returns:

a placement result containing list of bookie addresses for the ensemble.

Throws:

BKException.BKNotEnoughBookiesException - if not enough bookies available.

newEnsemble

public EnsemblePlacementPolicy.PlacementResult<java.util.List<BookieId>> newEnsemble(int ensembleSize,
                                                                                     int writeQuorumSize,
                                                                                     int ackQuorumSize,
                                                                                     java.util.Set<BookieId> excludeBookies,
                                                                                     ITopologyAwareEnsemblePlacementPolicy.Ensemble<BookieNode> parentEnsemble,
                                                                                     ITopologyAwareEnsemblePlacementPolicy.Predicate<BookieNode> parentPredicate)
                                                                              throws BKException.BKNotEnoughBookiesException

Description copied from interface: ITopologyAwareEnsemblePlacementPolicy

Create an ensemble with parent ensemble.

Parameters:

ensembleSize - ensemble size

writeQuorumSize - write quorum size

ackQuorumSize - ack quorum size

excludeBookies - exclude bookies

parentEnsemble - parent ensemble

Returns:

list of bookies forming the ensemble

Throws:

BKException.BKNotEnoughBookiesException

newEnsembleInternal

protected EnsemblePlacementPolicy.PlacementResult<java.util.List<BookieId>> newEnsembleInternal(int ensembleSize,
                                                                                                int writeQuorumSize,
                                                                                                int ackQuorumSize,
                                                                                                java.util.Set<BookieId> excludeBookies,
                                                                                                ITopologyAwareEnsemblePlacementPolicy.Ensemble<BookieNode> parentEnsemble,
                                                                                                ITopologyAwareEnsemblePlacementPolicy.Predicate<BookieNode> parentPredicate)
                                                                                         throws BKException.BKNotEnoughBookiesException

Throws:

BKException.BKNotEnoughBookiesException

replaceBookie

public EnsemblePlacementPolicy.PlacementResult<BookieId> replaceBookie(int ensembleSize,
                                                                       int writeQuorumSize,
                                                                       int ackQuorumSize,
                                                                       java.util.Map<java.lang.String,byte[]> customMetadata,
                                                                       java.util.List<BookieId> currentEnsemble,
                                                                       BookieId bookieToReplace,
                                                                       java.util.Set<BookieId> excludeBookies)
                                                                throws BKException.BKNotEnoughBookiesException

Description copied from interface: EnsemblePlacementPolicy

Choose a new bookie to replace bookieToReplace. If no bookie available in the cluster, BKException.BKNotEnoughBookiesException is thrown.

If 'enforceMinNumRacksPerWriteQuorum' config is enabled then the bookies belonging to default faultzone (rack) will be excluded while selecting bookies.

Parameters:

ensembleSize - the value of ensembleSize

writeQuorumSize - the value of writeQuorumSize

ackQuorumSize - the value of ackQuorumSize (added since 4.5)

customMetadata - the value of customMetadata. it is the same user defined metadata that user provides in BookKeeper.createLedger(int, int, int, BookKeeper.DigestType, byte[])

currentEnsemble - the value of currentEnsemble

bookieToReplace - bookie to replace

excludeBookies - bookies that should not be considered as candidate.

Returns:

a placement result containing the new bookie address.

Throws:

BKException.BKNotEnoughBookiesException

selectFromNetworkLocation

public BookieNode selectFromNetworkLocation(java.lang.String networkLoc,
                                            java.util.Set<Node> excludeBookies,
                                            ITopologyAwareEnsemblePlacementPolicy.Predicate<BookieNode> predicate,
                                            ITopologyAwareEnsemblePlacementPolicy.Ensemble<BookieNode> ensemble,
                                            boolean fallbackToRandom)
                                     throws BKException.BKNotEnoughBookiesException

Description copied from interface: ITopologyAwareEnsemblePlacementPolicy

Select a node from a given network location.

Parameters:

networkLoc - network location

excludeBookies - exclude bookies set

predicate - predicate to apply

ensemble - ensemble

fallbackToRandom - fallbackToRandom

Returns:

the selected bookie.

Throws:

BKException.BKNotEnoughBookiesException

selectFromNetworkLocation

public BookieNode selectFromNetworkLocation(java.lang.String networkLoc,
                                            java.util.Set<java.lang.String> excludeRacks,
                                            java.util.Set<Node> excludeBookies,
                                            ITopologyAwareEnsemblePlacementPolicy.Predicate<BookieNode> predicate,
                                            ITopologyAwareEnsemblePlacementPolicy.Ensemble<BookieNode> ensemble,
                                            boolean fallbackToRandom)
                                     throws BKException.BKNotEnoughBookiesException

Description copied from interface: ITopologyAwareEnsemblePlacementPolicy

Select a node from networkLoc rack excluding excludeBookies. If there isn't any node in 'networkLoc', then it will try to get a node from cluster excluding excludeRacks and excludeBookies. If fallbackToRandom is set to true then it will get a random bookie from cluster excluding excludeBookies if it couldn't find a bookie

Returns:

Throws:

BKException.BKNotEnoughBookiesException

selectFromNetworkLocation

public BookieNode selectFromNetworkLocation(java.util.Set<java.lang.String> excludeRacks,
                                            java.util.Set<Node> excludeBookies,
                                            ITopologyAwareEnsemblePlacementPolicy.Predicate<BookieNode> predicate,
                                            ITopologyAwareEnsemblePlacementPolicy.Ensemble<BookieNode> ensemble,
                                            boolean fallbackToRandom)
                                     throws BKException.BKNotEnoughBookiesException

It randomly selects a BookieNode that is not on the excludeRacks set, excluding the nodes in excludeBookies set. If it fails to find one, it selects a random BookieNode from the whole cluster.

Returns:

Throws:

BKException.BKNotEnoughBookiesException

selectRandomFromRack

protected BookieNode selectRandomFromRack(java.lang.String netPath,
                                          java.util.Set<Node> excludeBookies,
                                          ITopologyAwareEnsemblePlacementPolicy.Predicate<BookieNode> predicate,
                                          ITopologyAwareEnsemblePlacementPolicy.Ensemble<BookieNode> ensemble)
                                   throws BKException.BKNotEnoughBookiesException

Choose random node under a given network path.

Parameters:

netPath - network path

excludeBookies - exclude bookies

predicate - predicate to check whether the target is a good target.

ensemble - ensemble structure

Returns:

chosen bookie.

Throws:

BKException.BKNotEnoughBookiesException

selectRandom

protected java.util.List<BookieNode> selectRandom(int numBookies,
                                                  java.util.Set<Node> excludeBookies,
                                                  ITopologyAwareEnsemblePlacementPolicy.Predicate<BookieNode> predicate,
                                                  ITopologyAwareEnsemblePlacementPolicy.Ensemble<BookieNode> ensemble)
                                           throws BKException.BKNotEnoughBookiesException

Choose a random node from whole cluster.

Parameters:

numBookies - number bookies to choose

excludeBookies - bookies set to exclude.

ensemble - ensemble to hold the bookie chosen.

Returns:

the bookie node chosen.

Throws:

BKException.BKNotEnoughBookiesException

selectRandomInternal

protected java.util.List<BookieNode> selectRandomInternal(java.util.List<BookieNode> bookiesToSelectFrom,
                                                          int numBookies,
                                                          java.util.Set<Node> excludeBookies,
                                                          ITopologyAwareEnsemblePlacementPolicy.Predicate<BookieNode> predicate,
                                                          ITopologyAwareEnsemblePlacementPolicy.Ensemble<BookieNode> ensemble)
                                                   throws BKException.BKNotEnoughBookiesException

Throws:

BKException.BKNotEnoughBookiesException

registerSlowBookie

public void registerSlowBookie(BookieId bookieSocketAddress,
                               long entryId)

Description copied from interface: EnsemblePlacementPolicy

Register a bookie as slow so that it is tried after available and read-only bookies.

Parameters:

bookieSocketAddress - Address of bookie host

entryId - Entry ID that caused a speculative timeout on the bookie.

reorderReadSequence

public DistributionSchedule.WriteSet reorderReadSequence(java.util.List<BookieId> ensemble,
                                                         BookiesHealthInfo bookiesHealthInfo,
                                                         DistributionSchedule.WriteSet writeSet)

Description copied from interface: EnsemblePlacementPolicy

Reorder the read sequence of a given write quorum writeSet.

Specified by:

reorderReadSequence in interface EnsemblePlacementPolicy

Overrides:

reorderReadSequence in class TopologyAwareEnsemblePlacementPolicy

Parameters:

ensemble - Ensemble to read entries.

bookiesHealthInfo - Health info for bookies

writeSet - Write quorum to read entries. This will be modified, rather than allocating a new WriteSet.

Returns:

The read sequence. This will be the same object as the passed in writeSet.

reorderReadSequenceWithRegion

DistributionSchedule.WriteSet reorderReadSequenceWithRegion(java.util.List<BookieId> ensemble,
                                                            DistributionSchedule.WriteSet writeSet,
                                                            java.util.Map<java.lang.Integer,java.lang.String> writeSetWithRegion,
                                                            BookiesHealthInfo bookiesHealthInfo,
                                                            boolean regionAware,
                                                            java.lang.String myRegion,
                                                            int remoteNodeInReorderSequence)

This function orders the read sequence with a given region. For region-unaware policies (e.g. RackAware), we pass in false for regionAware and an empty myRegion. When this happens, any remote list will stay empty. The ordering is as follows (the R* at the beginning of each list item is only present for region aware policies). 1. available (local) bookies 2. R* a remote bookie (based on remoteNodeInReorderSequence 3. R* remaining (local) bookies 4. R* remaining remote bookies 5. read only bookies 6. slow bookies 7. unavailable bookies

Parameters:

ensemble - ensemble of bookies

writeSet - write set

writeSetWithRegion - write set with region information

bookiesHealthInfo - heuristics about health of boookies

regionAware - whether or not a region-aware policy is used

myRegion - current region of policy

remoteNodeInReorderSequence - number of local bookies to try before trying a remote bookie

Returns:

ordering of bookies to send read to

isEnsembleAdheringToPlacementPolicy

public EnsemblePlacementPolicy.PlacementPolicyAdherence isEnsembleAdheringToPlacementPolicy(java.util.List<BookieId> ensembleList,
                                                                                            int writeQuorumSize,
                                                                                            int ackQuorumSize)

Description copied from interface: EnsemblePlacementPolicy

returns AdherenceLevel if the Ensemble is strictly/softly/fails adhering to placement policy, like in the case of RackawareEnsemblePlacementPolicy, bookies in the writeset are from 'minNumRacksPerWriteQuorum' number of racks. And in the case of RegionawareEnsemblePlacementPolicy, check for minimumRegionsForDurability, reppRegionsToWrite, rack distribution within a region and other parameters of RegionAwareEnsemblePlacementPolicy. In ZoneAwareEnsemblePlacementPolicy if bookies in the writeset are from 'desiredNumOfZones' then it is considered as MEETS_STRICT if they are from 'minNumOfZones' then it is considered as MEETS_SOFT otherwise considered as FAIL.

Parameters:

ensembleList - list of BookieId of bookies in the ensemble

writeQuorumSize - writeQuorumSize of the ensemble

ackQuorumSize - ackQuorumSize of the ensemble

Returns:

areAckedBookiesAdheringToPlacementPolicy

public boolean areAckedBookiesAdheringToPlacementPolicy(java.util.Set<BookieId> ackedBookies,
                                                        int writeQuorumSize,
                                                        int ackQuorumSize)

Description copied from interface: EnsemblePlacementPolicy

Returns true if the bookies that have acknowledged a write adhere to the minimum fault domains as defined in the placement policy in use. Ex: In the case of RackawareEnsemblePlacementPolicy, bookies belong to at least 'minNumRacksPerWriteQuorum' number of racks.

Parameters:

ackedBookies - list of BookieId of bookies that have acknowledged a write.

writeQuorumSize - writeQuorumSize of the ensemble

ackQuorumSize - ackQuorumSize of the ensemble

Returns:

replaceToAdherePlacementPolicy

public EnsemblePlacementPolicy.PlacementResult<java.util.List<BookieId>> replaceToAdherePlacementPolicy(int ensembleSize,
                                                                                                        int writeQuorumSize,
                                                                                                        int ackQuorumSize,
                                                                                                        java.util.Set<BookieId> excludeBookies,
                                                                                                        java.util.List<BookieId> currentEnsemble)

Description copied from interface: EnsemblePlacementPolicy

Returns placement result. If the currentEnsemble is not adhering placement policy, returns new ensemble that adheres placement policy. It should be implemented so as to minify the number of bookies replaced.

Parameters:

ensembleSize - ensemble size

writeQuorumSize - writeQuorumSize of the ensemble

ackQuorumSize - ackQuorumSize of the ensemble

excludeBookies - bookies that should not be considered as targets

currentEnsemble - current ensemble

Returns:

a placement result