Package org.apache.bookkeeper.client

Class BookKeeperAdmin

  • All Implemented Interfaces:
    java.lang.AutoCloseable
    public class BookKeeperAdmin
extends java.lang.Object
implements java.lang.AutoCloseable
    Admin client for BookKeeper clusters.

    • Constructor Detail

      • BookKeeperAdmin

        public BookKeeperAdmin​(java.lang.String zkServers)
                throws java.io.IOException,
                       java.lang.InterruptedException,
                       BKException
        Constructor that takes in a ZooKeeper servers connect string so we know how to connect to ZooKeeper to retrieve information about the BookKeeper cluster. We need this before we can do any type of admin operations on the BookKeeper cluster.
        Parameters:
        zkServers - Comma separated list of hostname:port pairs for the ZooKeeper servers cluster.
        Throws:
        java.io.IOException - throws this exception if there is an error instantiating the ZooKeeper client.
        java.lang.InterruptedException - Throws this exception if there is an error instantiating the BookKeeper client.
        BKException - Throws this exception if there is an error instantiating the BookKeeper client.

      • BookKeeperAdmin

        public BookKeeperAdmin​(ClientConfiguration conf)
                throws java.io.IOException,
                       java.lang.InterruptedException,
                       BKException
        Constructor that takes in a configuration object so we know how to connect to ZooKeeper to retrieve information about the BookKeeper cluster. We need this before we can do any type of admin operations on the BookKeeper cluster.
        Parameters:
        conf - Client Configuration Object
        Throws:
        java.io.IOException - throws this exception if there is an error instantiating the ZooKeeper client.
        java.lang.InterruptedException - Throws this exception if there is an error instantiating the BookKeeper client.
        BKException - Throws this exception if there is an error instantiating the BookKeeper client.

      • BookKeeperAdmin

        public BookKeeperAdmin​(BookKeeper bkc,
                       StatsLogger statsLogger,
                       ClientConfiguration conf)
        Constructor that takes in a BookKeeper instance . This will be useful, when user already has bk instance ready.
        Parameters:
        bkc - - bookkeeper instance
        statsLogger - - stats logger

      • BookKeeperAdmin

        public BookKeeperAdmin​(BookKeeper bkc)

    • Method Detail

      • close

        public void close()
           throws java.lang.InterruptedException,
                  BKException
        Gracefully release resources that this client uses.
        Specified by:
        close in interface java.lang.AutoCloseable
        Throws:
        java.lang.InterruptedException - if there is an error shutting down the clients that this class uses.
        BKException

      • getAvailableBookies

        public java.util.Collection<BookieId> getAvailableBookies()
                                                   throws BKException
        Get a list of the available bookies.
        Returns:
        a collection of bookie addresses
        Throws:
        BKException

      • getAllBookies

        public java.util.Collection<BookieId> getAllBookies()
                                             throws BKException
        Get a list of all bookies including the not available ones.
        Returns:
        a collection of bookie addresses
        Throws:
        BKException

      • getReadOnlyBookies

        public java.util.Collection<BookieId> getReadOnlyBookies()
                                                  throws BKException
        Get a list of readonly bookies synchronously.
        Returns:
        a collection of bookie addresses
        Throws:
        BKException - if there are issues trying to read the list.

      • watchWritableBookiesChanged

        public void watchWritableBookiesChanged​(RegistrationClient.RegistrationListener listener)
                                 throws BKException
        Notify when the available list of bookies changes. Once registered, the listener will be notified when the list of available bookies changes.
        Parameters:
        listener - the listener to notify
        Throws:
        BKException

      • watchReadOnlyBookiesChanged

        public void watchReadOnlyBookiesChanged​(RegistrationClient.RegistrationListener listener)
                                 throws BKException
        Notify when the available list of read only bookies changes. Once registered, the listener will be notified when the list of available bookies changes.
        Parameters:
        listener - the listener to notify
        Throws:
        BKException

      • readEntries

        public java.lang.Iterable<LedgerEntry> readEntries​(long ledgerId,
                                                   long firstEntry,
                                                   long lastEntry)
                                            throws java.lang.InterruptedException,
                                                   BKException
        Read entries from a ledger synchronously. If the lastEntry is -1, it will read all the entries in the ledger from the firstEntry.
        Parameters:
        ledgerId -
        firstEntry -
        lastEntry -
        Returns:
        Throws:
        java.lang.InterruptedException
        BKException

      • getLedgersContainBookies

        public java.util.SortedMap<java.lang.Long,​LedgerMetadata> getLedgersContainBookies​(java.util.Set<BookieId> bookies)
                                                                                  throws java.lang.InterruptedException,
                                                                                         BKException
        Throws:
        java.lang.InterruptedException
        BKException

      • recoverBookieData

        public void recoverBookieData​(BookieId bookieSrc)
                       throws java.lang.InterruptedException,
                              BKException
        Synchronous method to rebuild and recover the ledger fragments data that was stored on the source bookie. That bookie could have failed completely and now the ledger data that was stored on it is under replicated. An optional destination bookie server could be given if we want to copy all of the ledger fragments data on the failed source bookie to it. Otherwise, we will just randomly distribute the ledger fragments to the active set of bookies, perhaps based on load. All ZooKeeper ledger metadata will be updated to point to the new bookie(s) that contain the replicated ledger fragments.
        Parameters:
        bookieSrc - Source bookie that had a failure. We want to replicate the ledger fragments that were stored there.
        Throws:
        java.lang.InterruptedException
        BKException

      • recoverBookieData

        public void recoverBookieData​(java.util.Set<BookieId> bookiesSrc)
                       throws java.lang.InterruptedException,
                              BKException
        Throws:
        java.lang.InterruptedException
        BKException

      • recoverBookieData

        public void recoverBookieData​(java.util.Set<BookieId> bookiesSrc,
                              boolean dryrun,
                              boolean skipOpenLedgers)
                       throws java.lang.InterruptedException,
                              BKException
        Throws:
        java.lang.InterruptedException
        BKException

      • recoverBookieData

        public void recoverBookieData​(java.util.Set<BookieId> bookiesSrc,
                              boolean dryrun,
                              boolean skipOpenLedgers,
                              boolean skipUnrecoverableLedgers)
                       throws java.lang.InterruptedException,
                              BKException
        Throws:
        java.lang.InterruptedException
        BKException

      • recoverBookieData

        public void recoverBookieData​(long lid,
                              java.util.Set<BookieId> bookiesSrc,
                              boolean dryrun,
                              boolean skipOpenLedgers)
                       throws java.lang.InterruptedException,
                              BKException
        Throws:
        java.lang.InterruptedException
        BKException

      • asyncRecoverBookieData

        public void asyncRecoverBookieData​(BookieId bookieSrc,
                                   AsyncCallback.RecoverCallback cb,
                                   java.lang.Object context)
        Async method to rebuild and recover the ledger fragments data that was stored on the source bookie. That bookie could have failed completely and now the ledger data that was stored on it is under replicated. An optional destination bookie server could be given if we want to copy all of the ledger fragments data on the failed source bookie to it. Otherwise, we will just randomly distribute the ledger fragments to the active set of bookies, perhaps based on load. All ZooKeeper ledger metadata will be updated to point to the new bookie(s) that contain the replicated ledger fragments.
        Parameters:
        bookieSrc - Source bookie that had a failure. We want to replicate the ledger fragments that were stored there.
        cb - RecoverCallback to invoke once all of the data on the dead bookie has been recovered and replicated.
        context - Context for the RecoverCallback to call.

      • asyncRecoverBookieData

        public void asyncRecoverBookieData​(java.util.Set<BookieId> bookieSrc,
                                   boolean dryrun,
                                   boolean skipOpenLedgers,
                                   boolean skipUnrecoverableLedgers,
                                   AsyncCallback.RecoverCallback cb,
                                   java.lang.Object context)

      • asyncRecoverBookieData

        public void asyncRecoverBookieData​(long lid,
                                   java.util.Set<BookieId> bookieSrc,
                                   boolean dryrun,
                                   boolean skipOpenLedgers,
                                   AsyncCallback.RecoverCallback callback,
                                   java.lang.Object context)
        Recover a specific ledger.
        Parameters:
        lid - ledger to recover
        bookieSrc - Source bookies that had a failure. We want to replicate the ledger fragments that were stored there.
        dryrun - dryrun the recover procedure.
        skipOpenLedgers - Skip recovering open ledgers.
        callback - RecoverCallback to invoke once all of the data on the dead bookie has been recovered and replicated.
        context - Context for the RecoverCallback to call.

      • formatEnsemble

        static java.lang.String formatEnsemble​(java.util.List<BookieId> ensemble,
                                       java.util.Set<BookieId> bookiesSrc,
                                       char marker)

      • replicateLedgerFragment

        public void replicateLedgerFragment​(LedgerHandle lh,
                                    LedgerFragment ledgerFragment,
                                    java.util.function.BiConsumer<java.lang.Long,​java.lang.Long> onReadEntryFailureCallback)
                             throws java.lang.InterruptedException,
                                    BKException
        Replicate the Ledger fragment to target Bookie passed.
        Parameters:
        lh - - ledgerHandle
        ledgerFragment - - LedgerFragment to replicate
        Throws:
        java.lang.InterruptedException
        BKException

      • format

        public static boolean format​(ServerConfiguration conf,
                             boolean isInteractive,
                             boolean force)
                      throws java.lang.Exception
        Format the BookKeeper metadata in zookeeper.
        Parameters:
        isInteractive - Whether format should ask prompt for confirmation if old data exists or not.
        force - If non interactive and force is true, then old data will be removed without prompt.
        Returns:
        Returns true if format succeeds else false.
        Throws:
        java.lang.Exception

      • initNewCluster

        public static boolean initNewCluster​(ServerConfiguration conf)
                              throws java.lang.Exception
        Initializes new cluster by creating required znodes for the cluster. If ledgersrootpath is already existing then it will error out.
        Parameters:
        conf -
        Returns:
        Throws:
        java.lang.Exception

      • nukeExistingCluster

        public static boolean nukeExistingCluster​(ServerConfiguration conf,
                                          java.lang.String ledgersRootPath,
                                          java.lang.String instanceId,
                                          boolean force)
                                   throws java.lang.Exception
        Nukes existing cluster metadata. But it does only if the provided ledgersRootPath matches with configuration's zkLedgersRootPath and provided instanceid matches with the cluster metadata. If force is mentioned then instanceid will not be validated.
        Parameters:
        conf -
        ledgersRootPath -
        instanceId -
        force -
        Returns:
        Throws:
        java.lang.Exception

      • initBookie

        public static boolean initBookie​(ServerConfiguration conf)
                          throws java.lang.Exception
        Initializes bookie, by making sure that the journalDir, ledgerDirs and indexDirs are empty and there is no registered Bookie with this BookieId.
        Parameters:
        conf -
        Returns:
        Throws:
        java.lang.Exception

      • listLedgers

        public java.lang.Iterable<java.lang.Long> listLedgers()
                                               throws java.io.IOException
        This method returns an iterable object for the list of ledger identifiers of the ledgers currently available.
        Returns:
        an iterable object for the list of ledger identifiers
        Throws:
        java.io.IOException - if the list of ledger identifiers cannot be read from the metadata store

      • getLedgerMetadata

        public LedgerMetadata getLedgerMetadata​(LedgerHandle lh)
        Returns:
        the metadata for the passed ledger handle

      • areEntriesOfLedgerStoredInTheBookie

        public static boolean areEntriesOfLedgerStoredInTheBookie​(long ledgerId,
                                                          BookieId bookieAddress,
                                                          LedgerManager ledgerManager)

      • areEntriesOfLedgerStoredInTheBookie

        public static boolean areEntriesOfLedgerStoredInTheBookie​(long ledgerId,
                                                          BookieId bookieAddress,
                                                          LedgerMetadata ledgerMetadata)

      • isEnsembleAdheringToPlacementPolicy

        public EnsemblePlacementPolicy.PlacementPolicyAdherence isEnsembleAdheringToPlacementPolicy​(java.util.List<BookieId> ensembleBookiesList,
                                                                                            int writeQuorumSize,
                                                                                            int ackQuorumSize)
        returns boolean value specifying if the ensemble of the segment is adhering to the ensemble placement policy for the given writeQuorumSize and ackQuorumSize.
        Parameters:
        ensembleBookiesList - ensemble of the segment
        writeQuorumSize - writeQuorumSize of the ledger
        ackQuorumSize - ackQuorumSize of the ledger
        Returns:
        true if the ledger is adhering to EnsemblePlacementPolicy

      • replaceNotAdheringPlacementPolicyBookie

        public java.util.Map<java.lang.Integer,​BookieId> replaceNotAdheringPlacementPolicyBookie​(java.util.List<BookieId> ensembleBookiesList,
                                                                                               int writeQuorumSize,
                                                                                               int ackQuorumSize)

      • asyncGetListOfEntriesOfLedger

        public java.util.concurrent.CompletableFuture<AvailabilityOfEntriesOfLedger> asyncGetListOfEntriesOfLedger​(BookieId address,
                                                                                                           long ledgerId)
        Makes async request for getting list of entries of ledger from a bookie and returns Future for the result.
        Parameters:
        address - BookieId of the bookie
        ledgerId - ledgerId
        Returns:
        returns Future

      • getCurrentAuditor

        public BookieId getCurrentAuditor()
                           throws java.io.IOException,
                                  java.lang.InterruptedException
        Throws:
        java.io.IOException
        java.lang.InterruptedException