Package com.tangosol.io.journal

Class FlashJournalRM

java.lang.Object
com.tangosol.io.journal.AbstractJournalRM
com.tangosol.io.journal.FlashJournalRM
All Implemented Interfaces:
Disposable, BinaryStoreManager, ClassLoaderAware, JournalMBean, Controllable, AutoCloseable
public class FlashJournalRM extends AbstractJournalRM
A FlashJournalRM manages disk I/O for temporary journal-based files, and acts as a shared resource for any number of journals that share a common configuration.

To use the Resource Manager, configure it using Dependency Injection by passing in the populated dependency object at instantiation. Once configured, start the Resource Manager via the AbstractJournalRM.start() method, and then obtain BinaryStore instances via the AbstractJournalRM.createBinaryStore() method. When a BinaryStore instance is no longer required, dispose of it via the Disposable interface, and similarly stop() or AbstractJournalRM.dispose() of the Resource Manager when it is no longer needed to ensure that any resources it allocated are cleaned up and released.

While the Resource Manager is running, there will be one or more files created in the specified temporary directory (or the default temporary directory if none is specified). The number of files is not related to the number of BinaryStore instances, but rather to the total amount of data that the Resource Manager is managing at any given time. Additionally, the Resource Manager uses three daemon threads to help manage the data: "Journal-Preparer", "Journal-Writer" and "Journal-Collector". These threads respectively lay out the data to be written, write the data, and perform asynchronous GC of the storage files.

The limits on the journal are as follows:

  • Values are limited by default to 64MB;
  • An individual file is limited by default to 2GB (and maximum 4GB);
  • A journal is composed of up to 512 files;
  • A journal is thus limited by default to 1TB, with a theoretical maximum of 2TB

Since:
Coherence 3.7
Author:
cp/cf 2010-06-10

  • Field Details

    • SHIFT_FILE_ID

      protected static final int SHIFT_FILE_ID
      The location of the file id within the ticket.
      See Also:

    • MASK_FILE_ID

      protected static final long MASK_FILE_ID
      The 64-bit bitmask for the file index. The 9 bits from 62-54.
      See Also:

    • SHIFT_OFFSET

      protected static final int SHIFT_OFFSET
      The location of the value-offset within the ticket.
      See Also:

    • SHIFT_OFFSET_RAW

      protected static final int SHIFT_OFFSET_RAW
      From an encoding point of view, the raw offset is shifted four bits less than necessary, because the four least significant bits of the offset are always zero (due to paragraph alignment).
      See Also:

    • MASK_OFFSET

      protected static final long MASK_OFFSET
      The 64-bit bitmask for the value offset. The 28 bits 53-26.

      Note that the least significant four bits of the offset MUST ALWAYS be zero, because they are NOT encoded into the ticket -- hence the 16-byte paragraph boundary for entries.

      See Also:

    • SHIFT_LENGTH

      protected static final int SHIFT_LENGTH
      The location of the value-length within the ticket.
      See Also:

    • MASK_LENGTH

      protected static final long MASK_LENGTH
      The 64-bit bitmask for the value length. The 26 bits 25-0.
      See Also:

    • TYPE_NAME

      protected static final String TYPE_NAME
      A simple name to be used as part of MBean ObjectNames.
      See Also:

    • WAIT_CONGESTION_ACTION

      protected static final FlashJournalRM.WaitCongestionPolicy WAIT_CONGESTION_ACTION
      The default congestion policy which blocks the client thread until awoken by the FlashJournalRM.WriterDaemon.

    • FAIL_CONGESTION_ACTION

      protected static final FlashJournalRM.FailCongestionPolicy FAIL_CONGESTION_ACTION
      A FlashJournalRM.CongestionPolicy which fails, by throwing an unchecked exception, when the file is deemed congested.

    • m_congestionPolicy

      protected volatile FlashJournalRM.CongestionPolicy m_congestionPolicy
      The FlashJournalRM.CongestionPolicy to be used when in a congested state.

    • m_dirTemp

      protected File m_dirTemp
      Directory to store the temporary files.

    • m_fLoggedDirectory

      protected boolean m_fLoggedDirectory
      Set to true after the directory location has been logged.

    • m_cbBacklog

      protected AtomicInteger m_cbBacklog
      Total amount of data enqueued to be written.

    • m_mapLocators

      protected ConcurrentMap<Long,Binary> m_mapLocators
      Map of locators to Binary values that have been queued to be written. A "locator" is a ticket without an encoded length.

    • m_pool

      protected FlashJournalRM.BufferPool m_pool
      The BufferPool of Buffer objects that are used to format data to append to the journal.

    • m_daemonPreparer

      protected FlashJournalRM.PreparerDaemon m_daemonPreparer
      The "rendering" thread that renders (copies and encrypts) the queued binaries into buffers to be written.

    • m_daemonWriter

      protected FlashJournalRM.WriterDaemon m_daemonWriter
      The "writing" thread that just does appends of prepared buffers to journal files.

  • Constructor Details

  • Method Details

    • stop

      public void stop()
      Hard-stop the controllable service. Use Controllable.shutdown() for normal service termination. Calling this method for a service that has already stopped has no effect.
      Specified by:
      stop in interface Controllable
      Overrides:
      stop in class AbstractJournalRM

    • getMaxTotalRam

      public long getMaxTotalRam()
      Determine the total amount, in bytes, of RAM that will be used for the Journal.

      This property is specific to a RamJournalRM implementation.

      Returns:
      the maximum number of bytes that will be allocated for Journal storage

    • isNioRam

      public boolean isNioRam()
      Determine if the journal will use direct buffers (NIO RAM) instead of on-heap buffers (byte arrays).

      This property is specific to a RamJournalRM implementation.

      Returns:
      true iff the journal is configured to use NIO "direct buffer" RAM

    • getMaxBacklogSize

      public int getMaxBacklogSize()
      Determine the maximum allowable size, in bytes, of the backlog; when the backlog reaches this level, writes are delayed until the backlog drops below its maximum.

      This property is specific to a FlashJournalRM implementation.

      Returns:
      the maximum allowable size of the backlog

    • getBacklogSize

      public int getBacklogSize()
      Determine the total size in bytes of the serialized values that have not yet been persisted to disk. This value is also referred to as the "backlog".

      This property is specific to a FlashJournalRM implementation.

      Returns:
      the number of bytes queued to be written

    • getBacklogCount

      public int getBacklogCount()
      Determine the number of serialized values that have not yet been persisted to disk.

      This property is specific to a FlashJournalRM implementation.

      Returns:
      the number of serialized values queued to be written

    • getBufferSize

      public int getBufferSize()
      Determine the size of the buffers used to write a chunk of data at a time to an underlying journal file.

      This property is specific to a FlashJournalRM implementation.

      Returns:
      the size, in bytes, of each buffer

    • getMaxPoolSize

      public long getMaxPoolSize()
      Determine the maximum size (in bytes) of the buffers that can be held by the pool. This is not a limit of how many buffers can be allocated, since some may be in use at any time, but rather how many will be held by the pool (i.e. recycled) as they are released.
      Returns:
      the maximum size in bytes of all of the buffers that the buffer pool can hold onto

    • getPoolSize

      public int getPoolSize()
      Determine the size (in bytes) of the buffers that are currently available in the pool. This is not a measurement of how many buffers are currently in use or how many have been allocated.

      This property is specific to a FlashJournalRM implementation.

      Returns:
      the total size in bytes of all of the buffers that are currently in the buffer pool

    • getHighFileCount

      public int getHighFileCount()
      Determine the high file count for this Journal. Compulsory compaction (GC) occurs when the journal file count reaches the high file count.

      This property is specific to a FlashJournalRM implementation.

      Returns:
      the high file count for this journal

    • getDirectoryFile

      public File getDirectoryFile()
      Determine the directory used to store the underlying disk files for the journal.
      Returns:
      the directory that is used to hold the journal files, or null if the default temporary directory is used

    • getPurgeDelayMillis

      public long getPurgeDelayMillis()
      Return the delay since the last touch of a tmp file until it is eligible for purging. Tmp files are purged at startup of the FlashJournalRM to clean up tmp files from a previous instance of the FlashJournalRM.
      Returns:
      the delay in milliseconds

    • getName

      protected String getName()
      Returns the name value to be used in MBean Objectname key "name".
      Specified by:
      getName in class AbstractJournalRM
      Returns:
      the name

    • getWriterTimeout

      protected long getWriterTimeout()
      Return the number of milliseconds the Writer daemon can be unresponsive prior to considering it timed out.
      Returns:
      the number of milliseconds the Writer can be unresponsive

    • getDependencies

      public FlashJournalRM.Dependencies getDependencies()
      Return the Dependencies object.
      Overrides:
      getDependencies in class AbstractJournalRM
      Returns:
      the journal dependencies object

    • cloneDependencies

      public FlashJournalRM.DefaultDependencies cloneDependencies(AbstractJournalRM.Dependencies deps)
      Clone the Dependencies object.
      Specified by:
      cloneDependencies in class AbstractJournalRM
      Parameters:
      deps - the populated Dependencies object
      Returns:
      the cloned Dependencies object

    • onDependencies

      protected void onDependencies(AbstractJournalRM.Dependencies deps)
      Process the Dependencies after they have been injected into this object.
      Overrides:
      onDependencies in class AbstractJournalRM
      Parameters:
      deps - the populated Dependencies object

    • startThreads

      protected void startThreads()
      Create and start the various threads used by this Journal Resource Manager.
      Overrides:
      startThreads in class AbstractJournalRM

    • purgeOldTempFiles

      protected void purgeOldTempFiles()
      Delete any old temp files that weren't cleaned up properly by a previous are configured to be "deleted on exit", it is possible that the temp files would not have been deleted if the JVM crashed or if there was a fault at the physical machine level.

    • stopThreads

      protected void stopThreads()
      Stop the various threads used by this Journal Resource Manager.
      Overrides:
      stopThreads in class AbstractJournalRM

    • calculateCurrentLoadFactor

      protected double calculateCurrentLoadFactor()
      Calculate current load factor for compaction (GC).
      Overrides:
      calculateCurrentLoadFactor in class AbstractJournalRM
      Returns:
      calculated load factor

    • shouldNotifyCollector

      protected boolean shouldNotifyCollector()
      Check if the Collector daemon needs to be notified to come out of sleep
      Overrides:
      shouldNotifyCollector in class AbstractJournalRM
      Returns:
      true if collector daemon should be notified

    • getDescription

      protected String getDescription()
      Format the object attributes into a String for inclusion in the String returned from the AbstractJournalRM.toString() method.
      Overrides:
      getDescription in class AbstractJournalRM
      Returns:
      a comma-delimited String listing the attributes of this object in the form "attribute=value"

    • getJournalFile

      protected FlashJournalRM.JournalFile getJournalFile(int nFileId)
      Obtain a JournalFile by its ID.
      Overrides:
      getJournalFile in class AbstractJournalRM
      Parameters:
      nFileId - the file id (0-511)
      Returns:
      the JournalFile for that file id, or null if there is currently no JournalFile for that file id

    • instantiateJournalFile

      protected FlashJournalRM.JournalFile instantiateJournalFile(int nFile)
      Factory: Instantiate a JournalFile or subclass thereof.
      Specified by:
      instantiateJournalFile in class AbstractJournalRM
      Parameters:
      nFile - the file number in the range 0-511 (inclusive)
      Returns:
      a new JournalFile instance

    • instantiateBufferPool

      protected FlashJournalRM.BufferPool instantiateBufferPool()
      Factory: Instantiate a pool of buffers.
      Returns:
      a BufferPool or subclass thereof

    • getBufferPool

      protected FlashJournalRM.BufferPool getBufferPool()
      Obtain the BufferPool instance.
      Returns:
      the BufferPool for this FlashJournalRM

    • instantiatePreparerDaemon

      protected FlashJournalRM.PreparerDaemon instantiatePreparerDaemon()
      Factory: Instantiate a PreparerDaemon.
      Returns:
      a PreparerDaemon or a sub-class thereof

    • getPreparerDaemon

      protected FlashJournalRM.PreparerDaemon getPreparerDaemon()
      Obtain the PreparerDaemon instance.
      Returns:
      the PreparerDaemon for this FlashJournalRM

    • instantiateWriterDaemon

      protected FlashJournalRM.WriterDaemon instantiateWriterDaemon()
      Factory: Instantiate a WriterDaemon.
      Returns:
      a WriterDaemon or a sub-class thereof

    • getWriterDaemon

      protected FlashJournalRM.WriterDaemon getWriterDaemon()
      Obtain the WriterDaemon instance.
      Returns:
      the WriterDaemon for this FlashJournalRM

    • getEvacuationMask

      protected long getEvacuationMask()
      Obtain the bit mask that is used to identify tickets that need to be evacuated for a particular file id.
      Specified by:
      getEvacuationMask in class AbstractJournalRM
      Returns:
      the bit mask for the file index within a ticket and whatever other bits are necessary to ensure that a ticket refers to that file

    • extractFileId

      protected int extractFileId(long lTicket)
      Extract a file index (a file ID) from the passed ticket.
      Specified by:
      extractFileId in class AbstractJournalRM
      Parameters:
      lTicket - a "ticket" that was created by the Journal
      Returns:
      the file index for the value represented by that ticket

    • extractOffset

      protected long extractOffset(long lTicket)
      Extract a file offset from the passed ticket.
      Specified by:
      extractOffset in class AbstractJournalRM
      Parameters:
      lTicket - a "ticket" that was created by the Journal
      Returns:
      the file offset for the value represented by that ticket

    • extractLength

      protected int extractLength(long lTicket)
      Extract a file offset from the passed ticket.
      Specified by:
      extractLength in class AbstractJournalRM
      Parameters:
      lTicket - a "ticket" that was created by the Journal
      Returns:
      the file offset for the value represented by that ticket

    • encodeTicket

      protected long encodeTicket(int nFile, long of, int cb)
      Encode a file number, offset and length into a "ticket".
      Specified by:
      encodeTicket in class AbstractJournalRM
      Parameters:
      nFile - a file number (aka a file index)
      of - a value offset
      cb - a value length
      Returns:
      a "ticket" containing the file number, offset and length