Package com.tangosol.io.journal

Class AbstractJournalRM

java.lang.Object
com.tangosol.io.journal.AbstractJournalRM
All Implemented Interfaces:
Disposable, BinaryStoreManager, ClassLoaderAware, JournalMBean, Controllable, AutoCloseable
Direct Known Subclasses:
FlashJournalRM, RamJournalRM
public abstract class AbstractJournalRM extends Object implements Controllable, Disposable, BinaryStoreManager, JournalMBean
The AbstractJournalRM represents a Journal Resource Manager that manages storage for journaled data, and acts as a shared resource for any number of journals that share a common configuration. This implementation is intended to support both file-based and memory-based implementations.

The limits on the journal are as follows:

  • Values are limited by default to 64MB;
  • An individual file is limited by default to 2GB (and maximum 4GB);
  • A journal is composed of up to 512 files;
  • A journal is thus limited by default to 1TB, with a theoretical maximum of 2TB.

Since:
Coherence 3.7
Author:
cp/cf 2010-06-23

  • Field Details

    • STATE_INITIAL

      protected static final int STATE_INITIAL
      State: Initial (unconfigured).
      See Also:

    • STATE_CONFIGURED

      protected static final int STATE_CONFIGURED
      State: Configured (ready to be started).
      See Also:

    • STATE_RUNNING

      protected static final int STATE_RUNNING
      State: Running (able to be shut down or stopped).
      See Also:

    • STATE_STOPPING

      protected static final int STATE_STOPPING
      State: Stopping.
      See Also:

    • STATE_STOPPED

      protected static final int STATE_STOPPED
      State: Stopped.
      See Also:

    • SHIFT_COMPACT_FLAG

      protected static final int SHIFT_COMPACT_FLAG
      The location of the compact flag within the ticket.
      See Also:

    • MASK_COMPACT_FLAG

      protected static final long MASK_COMPACT_FLAG
      The 64-bit bitmask for the compact flag.

      The compact flag is the bit that signifies whether the ticket contains the actual value encoded into the long, versus a "proper" journal file ticket.

      See Also:

    • SHIFT_COMPACT_LENGTH

      protected static final int SHIFT_COMPACT_LENGTH
      The location of the compact length within the ticket.
      See Also:

    • MASK_COMPACT_LENGTH

      protected static final long MASK_COMPACT_LENGTH
      The 64-bit bitmask for the compact length.
      See Also:

    • DEDUPE_ENABLED

      protected static final boolean DEDUPE_ENABLED
      Whether de-duping (interning) the byte arrays held by the BinaryRadixTree should be requested.

    • m_nState

      protected int m_nState
      The current state of the Resource Manager; one of the STATE_* constants.

    • m_dependencies

      protected AbstractJournalRM.Dependencies m_dependencies
      The configuration used by this instance.

    • m_loader

      protected ClassLoader m_loader
      The ClassLoader supplied to this Controllable instance. (Controllable extends ClassLoaderAware.)

    • m_dflHighestLoadFactor

      protected double m_dflHighestLoadFactor
      The highest percentage (as a double in the range 0.00 to 1.00) of the configured capacity that this journal has reached. In other words, this is a high water mark of how full the journal has ever gotten.

    • m_dblFileSizeFactor

      protected double m_dblFileSizeFactor
      The size of the file as a ratio to the largest possible file size. Used to tune the aggressiveness of the collection.

    • m_setJournals

      protected final Set<AbstractJournalRM.JournalImpl> m_setJournals
      Registry of JournalImpl objects.

    • m_ajournalfile

      protected final AbstractJournalRM.JournalFile[] m_ajournalfile
      Registry of JournalFile objects indexed by file id.

    • f_ajournalfileLock

      protected final Lock f_ajournalfileLock
      The lock that should be used to control concurrent access to JournalFile objects.

    • f_ajournalfileCondition

      protected final Condition f_ajournalfileCondition
      A condition associated with a synchronization lock above that is used to wait for notification and to notify waiting threads.

    • m_cJournalFiles

      protected volatile int m_cJournalFiles
      The number of JournalFiles that currently exist.

    • m_jrnlfile

      protected volatile AbstractJournalRM.JournalFile m_jrnlfile
      The current JournalFile.

    • m_daemonCollector

      protected AbstractJournalRM.CollectorDaemon m_daemonCollector
      The "garbage collector" thread that determines when journal files are ripe to be evacuated and deleted.

    • f_journalRMLock

      protected final Lock f_journalRMLock
      The lock that should be used to control concurrent access to this Journal Resource Manager.

    • f_journalRMCondition

      protected final Condition f_journalRMCondition
      A condition associated with a synchronization lock above that is used to wait for notification and to notify waiting threads.

  • Constructor Details

  • Method Details

    • configure

      public void configure(XmlElement xml)
      Configure the dependencies object given the xml configuration.
      Specified by:
      configure in interface Controllable
      Parameters:
      xml - the XML configuration for the object

    • start

      public void start()
      Start the controllable service.

      This method should only be called once per the life cycle of the Controllable service. This method has no affect if the service is already running.

      Specified by:
      start in interface Controllable

    • isRunning

      public boolean isRunning()
      Determine whether or not the controllable service is running. This method returns false before a service is started, while the service is starting, while a service is shutting down and after the service has stopped. It only returns true after completing its start processing and before beginning its shutdown processing.
      Specified by:
      isRunning in interface Controllable
      Returns:
      true if the service is running; false otherwise

    • shutdown

      public void shutdown()
      Stop the controllable service. This is a controlled shut-down, and is preferred to the Controllable.stop() method.

      This method should only be called once per the life cycle of the controllable service. Calling this method for a service that has already stopped has no effect.

      Specified by:
      shutdown in interface Controllable

    • stop

      public void stop()
      Hard-stop the controllable service. Use Controllable.shutdown() for normal service termination. Calling this method for a service that has already stopped has no effect.
      Specified by:
      stop in interface Controllable

    • getContextClassLoader

      public ClassLoader getContextClassLoader()
      Retrieve the context ClassLoader for this object. The context ClassLoader is provided by the creator of the object for use by the object when loading classes and resources.
      Specified by:
      getContextClassLoader in interface ClassLoaderAware
      Returns:
      the context ClassLoader for this object
      See Also:

    • setContextClassLoader

      public void setContextClassLoader(ClassLoader loader)
      Specify the context ClassLoader for this object. The context ClassLoader can be set when the object is created, and allows the creator to provide the appropriate class loader to be used by the object when when loading classes and resources.
      Specified by:
      setContextClassLoader in interface ClassLoaderAware
      Parameters:
      loader - the context ClassLoader for this object

    • dispose

      public void dispose()
      Invoked when all resources owned by the implementer can safely be released.

      Once disposed of the object should no longer be considered to be usable.

      Note the Disposable interface is compatible with try-with-resources which will automatically invoke this method.

      Specified by:
      dispose in interface Disposable

    • close

      public void close()
      Deprecated.
      use the Disposable interface
      Previous to the Disposable interface, clean-up was performed by invoking a close method via reflection.
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Disposable

    • getMaxJournalFilesNumber

      public int getMaxJournalFilesNumber()
      Determine the maximum number of Journal files.
      Specified by:
      getMaxJournalFilesNumber in interface JournalMBean
      Returns:
      the maximum number of simultaneous Journal files that will be used to store the contents of this Journal

    • getBinaryStoreCount

      public int getBinaryStoreCount()
      Determine the number of active BinaryStore objects that are using this Journal.
      Specified by:
      getBinaryStoreCount in interface JournalMBean
      Returns:
      the number of BinaryStore objects that are currently active

    • getFileCount

      public int getFileCount()
      Determine the number of Journal files used by this Journal.
      Specified by:
      getFileCount in interface JournalMBean
      Returns:
      the number of files for this journal

    • getTotalDataSize

      public long getTotalDataSize()
      Determine the amount of data currently stored by this Journal. This value does not include data that have been released.
      Specified by:
      getTotalDataSize in interface JournalMBean
      Returns:
      the number of bytes for this Journal used to store Binary values that have not been released

    • getTotalFileSize

      public long getTotalFileSize()
      Determine the number of bytes in the Journal files for this Journal. This number could include released data that has not yet been compacted.
      Specified by:
      getTotalFileSize in interface JournalMBean
      Returns:
      the total size of all Journal files for this Journal

    • getMaxValueSize

      public int getMaxValueSize()
      Determine the maximum allowable size, in bytes, of a Binary value.
      Specified by:
      getMaxValueSize in interface JournalMBean
      Returns:
      the maximum allowable size for a value to write

    • getMaxFileSize

      public long getMaxFileSize()
      Determine the maximum allowable size, in bytes, of each individual file used to hold Journal data. This is not the total size limit of the Journal, since the Journal may use a number of files to store its data.
      Specified by:
      getMaxFileSize in interface JournalMBean
      Returns:
      the maximum allowable size for an underlying file

    • getCollectorLoadFactor

      public double getCollectorLoadFactor()
      Determine the load factor threshold at which files become eligible for garbage collection (compaction). The load factor threshold is the ratio between the portion of the file used for live data and the total size of the file, and is expressed as a double value in the interval (0.0, 1.0).

      When the portion of released data exceeds (1 - threshold), the file becomes eligible for compaction (garbage collection).

      Specified by:
      getCollectorLoadFactor in interface JournalMBean
      Returns:
      the compaction factor threshold between 0.0 and 1.0

    • getCurrentCollectorLoadFactor

      public double getCurrentCollectorLoadFactor()
      Determine the current load factor threshold for the journal.
      Specified by:
      getCurrentCollectorLoadFactor in interface JournalMBean
      Returns:
      the current load factor threshold between 0.0 and 1.0

    • getHighestLoadFactor

      public double getHighestLoadFactor()
      Determine the high-water mark load factor for the entire journal. The load factor is the ratio of the highest number of journal files that have concurrently existed to the maximum number of journal files that can concurrently exist.
      Specified by:
      getHighestLoadFactor in interface JournalMBean
      Returns:
      the high-water journal load factor in the range 0.00 to 1.00

    • getCompactionCount

      public int getCompactionCount()
      Determine the number of compactions (garbage collections) performed by this journal.
      Specified by:
      getCompactionCount in interface JournalMBean
      Returns:
      the number of compactions

    • getExhaustiveCompactionCount

      public int getExhaustiveCompactionCount()
      Determine the total number of exhaustive compactions performed by the collector. An exhaustive compaction is performed when there appears to be no memory available, however the garbage collector may have the ability reclaim memory via compaction.
      Specified by:
      getExhaustiveCompactionCount in interface JournalMBean
      Returns:
      number of exhaustive compactions executed

    • getExhaustiveCompactionTime

      public long getExhaustiveCompactionTime()
      Return the total amount time spent performing exhaustive compaction.
      Specified by:
      getExhaustiveCompactionTime in interface JournalMBean
      Returns:
      the total amount time spent performing exhaustive compaction

    • resetStatistics

      public void resetStatistics()
      Reset the journal statistics.
      Specified by:
      resetStatistics in interface JournalMBean

    • compact

      public void compact(boolean fRegular)
      Execute either a regular or exhaustive compaction on the journal based on the provided parameter. The compaction is performed asynchronous to the invocation of this operation.

      An exhaustive compaction is only executed if sufficient memory has been released or sufficient time has elapsed since the previous exhaustive compaction.

      Specified by:
      compact in interface JournalMBean
      Parameters:
      fRegular - whether the compaction should be regular; false for an exhaustive compaction

    • createBinaryStore

      public JournalBinaryStore createBinaryStore()
      Create a new BinaryStore that will journal its information using this Journal Resource Manager.
      Specified by:
      createBinaryStore in interface BinaryStoreManager
      Returns:
      a new instance of JournalBinaryStore that uses this Journal Resource Manager for storage

    • destroyBinaryStore

      public void destroyBinaryStore(BinaryStore store)
      Lifecycle method: Destroy a BinaryStore previously created by this manager.
      Specified by:
      destroyBinaryStore in interface BinaryStoreManager
      Parameters:
      store - a BinaryStore object previously created by this manager

    • createJournal

      public Journal createJournal(Journal.JournalConsumer consumer)
      Create a new Journal that exposes the storage capabilities of this Journal Resource Manager.
      Parameters:
      consumer - the JournalConsumer that will be using the Journal
      Returns:
      a new instance of Journal that uses this Journal Resource Manager for storage

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • getDependencies

      public AbstractJournalRM.Dependencies getDependencies()
      Return the Dependencies object.
      Returns:
      the journal dependencies object

    • cloneDependencies

      public abstract AbstractJournalRM.DefaultDependencies cloneDependencies(AbstractJournalRM.Dependencies deps)
      Clone the Dependencies object.
      Parameters:
      deps - the populated Dependencies object
      Returns:
      the cloned Dependencies object

    • onDependencies

      protected void onDependencies(AbstractJournalRM.Dependencies deps)
      Process the Dependencies after they have been injected into this object.
      Parameters:
      deps - the populated Dependencies object

    • getState

      protected int getState()
      Determine the current state of the AbstractJournalRM.
      Returns:
      one of the STATE_* constants

    • setState

      protected void setState(int nState)
      Modify the current state of the AbstractJournalRM.
      Parameters:
      nState - one of the STATE_* constants

    • startThreads

      protected void startThreads()
      Create and start the various threads used by this Journal Resource Manager.

    • stopThreads

      protected void stopThreads()
      Stop the various threads used by this Journal Resource Manager.

    • calculateCurrentLoadFactor

      protected double calculateCurrentLoadFactor()
      Calculate current load factor for compaction (GC).
      Returns:
      calculated load factor

    • getStateDescription

      protected static String getStateDescription(int nState)
      Format the passed state identifier into a human-readable string.
      Parameters:
      nState - a state identifier
      Returns:
      a human-readable string for the state

    • getDescription

      protected String getDescription()
      Format the object attributes into a String for inclusion in the String returned from the toString() method.
      Returns:
      a comma-delimited String listing the attributes of this object in the form "attribute=value"

    • getMaxJournalFiles

      protected int getMaxJournalFiles()
      Determine the maximum number of journal files.
      Returns:
      the maximum number of simultaneous journal files that will be used to store the contents of this journal

    • getMaxJournalSize

      protected long getMaxJournalSize()
      Return the total amount of memory, in bytes, that can be allocated for Journal storage.
      Returns:
      the maximum number of bytes that can be allocated for Journal storage

    • getMinCollectorSleepMillis

      protected long getMinCollectorSleepMillis()
      Determine the minimum sleep cycle between collections.
      Returns:
      the minimum number of milliseconds to sleep between collections

    • getMaxCollectorSleepMillis

      protected long getMaxCollectorSleepMillis()
      Determine the maximum sleep cycle between collections.
      Returns:
      the maximum number of milliseconds to sleep between collections

    • isDedupEnabled

      protected boolean isDedupEnabled()
      Determine if this journal resource manager should de-dup the keys that its JournalBinaryStore instances are managing.
      Returns:
      true iff this journal resource manager should routinely cause its JournalBinaryStore instances to de-dup their keys

    • isSingleEvacuation

      protected boolean isSingleEvacuation()
      Determine if this journal resource manager should only evacuate one single JournalFile (at the most) per collection cycle.
      Returns:
      true iff this journal resource manager should only evacuate the emptiest JournalFile instead of all the JournalFile instances that qualify for evacuation

    • getSurvivorFile

      protected AbstractJournalRM.JournalFile getSurvivorFile()
      Return a AbstractJournalRM.JournalFile based on the context of the calling thread or null if it is determined that an exhaustive evacuation will not reclaim sufficient memory. This method should only be invoked in a "depleted" state when:
      • the maximum number of files is used
      • all those files are non-appending
      When called on a client thread, any operation against this file will block until the full compaction is performed. There is no guarantee that compaction results in available memory thus an out of memory condition may still occur.

      When called on the CollectorDaemon thread this method returns a normal JournalFile, with the only exception that it will be in excess to the maximum number of permitted journal files and used exclusively by the CollectorDaemon to perform evacuation in a depleted state.

      Returns:
      either a JournalFile whose usage triggers a full collection or a JournalFile used exclusively by the CollectorDaemon. null may be returned if it is determined that an exhaustive evacuation will not reclaim sufficient memory

    • registerMBean

      protected void registerMBean()
      Register an MBean for this JournalRM.

    • unregisterMBean

      protected void unregisterMBean()
      Unregister an MBean for this JournalRM.

    • instantiateJournal

      protected AbstractJournalRM.JournalImpl instantiateJournal()
      Factory: Instantiate a JournalImpl object.
      Returns:
      a JournalImpl instance or subclass thereof

    • registerJournal

      protected void registerJournal(AbstractJournalRM.JournalImpl journal)
      Register a Journal that uses this Journal Resource Manager.
      Parameters:
      journal - an instance of JournalImpl created by this Journal Resource Manager

    • unregisterJournal

      protected void unregisterJournal(AbstractJournalRM.JournalImpl journal)
      Register a Journal that uses this Journal Resource Manager.
      Parameters:
      journal - an instance of JournalImpl created by this Journal Resource Manager

    • iterateJournals

      protected Iterator<AbstractJournalRM.JournalImpl> iterateJournals()
      Iterate through the Journal instances that have been created by this Journal Resource Manager but have not been disposed of yet.
      Returns:
      an Iterator of the current JournalImpl instances

    • getJournalFile

      protected AbstractJournalRM.JournalFile getJournalFile(int nFileId)
      Obtain a JournalFile by its ID.
      Parameters:
      nFileId - the file id (0-511)
      Returns:
      the JournalFile for that file id, or null if there is currently no JournalFile for that file id

    • ensureCurrentJournalFile

      protected AbstractJournalRM.JournalFile ensureCurrentJournalFile()
      Obtain the current JournalFile that is being appended to, creating a new one if necessary.
      Returns:
      the current JournalFile, or null if the storage is full and no new JournalFile can be created as a result

    • iterateJournalFiles

      protected Iterator<AbstractJournalRM.JournalFile> iterateJournalFiles()
      Iterate all of the JournalFile objects.
      Returns:
      an Iterator of the existing JournalFile instances

    • instantiateJournalFile

      protected abstract AbstractJournalRM.JournalFile instantiateJournalFile(int nFile)
      Factory: Instantiate a JournalFile or subclass thereof.
      Parameters:
      nFile - the file number in the range 0-511 (inclusive)
      Returns:
      a new JournalFile instance

    • instantiateCollectorDaemon

      protected AbstractJournalRM.CollectorDaemon instantiateCollectorDaemon()
      Factory: Instantiate a CollectorDaemon.
      Returns:
      a CollectorDaemon or a sub-class thereof

    • getCollectorDaemon

      protected AbstractJournalRM.CollectorDaemon getCollectorDaemon()
      Obtain the CollectorDaemon instance.
      Returns:
      the CollectorDaemon for this Journal Resource Manager

    • isCompact

      protected boolean isCompact(long lTicket)
      Determine if the ticket is in the compact form.
      Parameters:
      lTicket - a "ticket" that was created by the Journal
      Returns:
      true iff the ticket is a compact ticket

    • getEvacuationMask

      protected abstract long getEvacuationMask()
      Obtain the bit mask that is used to identify tickets that need to be evacuated for a particular file id.
      Returns:
      the bit mask for the file index within a ticket and whatever other bits are necessary to ensure that a ticket refers to that file

    • extractFileId

      protected abstract int extractFileId(long lTicket)
      Extract a file index (a file ID) from the passed ticket.
      Parameters:
      lTicket - a "ticket" that was created by the Journal
      Returns:
      the file index for the value represented by that ticket

    • extractOffset

      protected abstract long extractOffset(long lTicket)
      Extract a file offset from the passed ticket.
      Parameters:
      lTicket - a "ticket" that was created by the Journal
      Returns:
      the file offset for the value represented by that ticket

    • extractLength

      protected abstract int extractLength(long lTicket)
      Extract a file offset from the passed ticket.
      Parameters:
      lTicket - a "ticket" that was created by the Journal
      Returns:
      the file offset for the value represented by that ticket

    • encodeTicket

      protected abstract long encodeTicket(int nFile, long of, int cb)
      Encode a file number, offset and length into a "ticket".
      Parameters:
      nFile - a file number (aka a file index)
      of - a value offset
      cb - a value length
      Returns:
      a "ticket" containing the file number, offset and length

    • encodeTicket

      protected long encodeTicket(ByteSequence bin)
      Encode a Binary into a "ticket".
      Parameters:
      bin - a ByteSequence with (length() <= 7)
      Returns:
      a compact "ticket" containing the passed Binary value

    • extractBinary

      protected Binary extractBinary(long lTicket)
      Extract a Binary from the passed ticket.
      Parameters:
      lTicket - a compact "ticket" that was created by the Journal
      Returns:
      the Binary encoded into the ticket

    • getTicketDescription

      protected String getTicketDescription(long lTicket)
      Format the information encoded in a ticket into a debug string.
      Parameters:
      lTicket - the ticket value
      Returns:
      a description of the contents of the ticket value

    • shouldNotifyCollector

      protected boolean shouldNotifyCollector()
      Check if the Collector daemon needs to be notified to come out of sleep
      Returns:
      true if collector daemon should be notified

    • getCluster

      protected Cluster getCluster()
      Returns the Cluster for which the AbstractJournalRM is storing data.
      Returns:
      the Cluster

    • getGuardian

      protected Guardian getGuardian()
      Return the Guardian associated with the Cluster or null if absent.
      Returns:
      the Guardian associated with the Cluster or null

    • getCollectorTimeout

      protected long getCollectorTimeout()
      Return the number of milliseconds the Collector can be unresponsive prior to considering the Collector timed out.
      Returns:
      the number of milliseconds the Collector can be unresponsive

    • getName

      protected abstract String getName()
      Returns the name value to be used in MBean Objectname key "name".
      Returns:
      the name