Package org.objectweb.howl.log

Class LogFileManager

  • class LogFileManager
extends LogObject
    Manage a set of log files. This class implements methods that can be called by LogBufferManager to obtain a LogFile for logger IO and to signal the LogFileManager when new buffers are being initialized. LogFileManager manages log files according to implementation specific policies. Some LogFileManagers may use a circular file policy while others may use a set of files. The most simple implementations will use a single file and allow it to grow as needed. QUESTION: do we need multiple implementations, or can we deal with different policies in this one class using configuration?

    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description
      (package private) class  LogFileManager.EventManager
      helper thread used invoke LogEventListener when log overflow (or other event) is about to occur.

    • Field Summary

      Fields 
      Modifier and Type Field Description
      (package private) long activeMark
      The log key for the oldest active entry in the log.
      (package private) boolean automark
      indicates whether log files will be marked automatically.
      (package private) byte[] autoMarkOff
      data written to log when autoMark is turned off
      (package private) byte[] autoMarkOn
      data written to log when autoMark is turned on
      (package private) LogBufferManager bmgr
      LogBufferManager used by methods that put() records to the log.
      (package private) byte[] crlf
      end of line for log records to make logs readable in text editors.
      (package private) long currentKey
      last key returned by put().
      (package private) LogFile currentLogFile  
      (package private) int event
      event to be processed by the eventManagerThread.
      private LogEventListener eventListener
      The LogEventListener registered by the application that owns the logger.
      (package private) java.lang.Object eventManagerLock
      Monitor used by EventManager thread to wait for events that need to be notified.
      (package private) java.lang.Thread eventManagerThread
      Thread used by LogFileManager to send events to the registered LogEventListener.
      (package private) byte[][] fileHeader
      LogFile header record.
      (package private) java.nio.ByteBuffer fileHeaderBB
      ByteBuffer wrapper for fileHeader to facilitate conversion of numeric information to byte[] format.
      private java.lang.Object fileManagerLock
      lock controlling access to LogFile.
      (package private) LogFile[] fileSet
      set of LogFile objects associated with the physical log files.
      private boolean initComplete
      Indicates that LogFileManager initialization is complete.
      private long initialKey
      the first log key generated by this instance of the Logger.
      (package private) int lfIndex
      workerID to current entry in fileSet[]
      (package private) static int LOG_OVERFLOW_EVENT
      Event types for event manager thread.
      private long lowestSafeLogKey
      The logKey to be used by LogEventListener.logOverflowNotification when moving records forward into the current log file.
      (package private) byte[][] markRecord
      MARK control Record.
      (package private) java.nio.ByteBuffer markRecordBB
      ByteBuffer wrapper for markRecord to facilitate conversion of numeric information to byte[] format.
      (package private) int maxBlocksPerFile
      maximum number of blocks to store in each LogFile.
      private int overflowNotificationCount
      Number of times log overflow event was notified.
      (package private) boolean restartAutoMark
      The automark value restored during log file initialization.

    • Constructor Summary

      Constructors 
      Constructor Description
      LogFileManager​(Configuration config)
      construct LogFileManager with Configuration supplied by caller.

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      (package private) void close()
      Gracefully close the log files.
      (package private) void closeBufferManager()
      Write a CLOSE record and shut down the buffer manager if we have one.
      private void detectLogOverflow​(int bsn)
      Detect pending Log Overflow and notify event listener.
      (package private) long getHighMark()
      Returns the highMark from the LogFile that is currently being written.
      (package private) LogFile getLogFileForMark​(long mark)
      Returns the LogFile that contains the requested mark .
      (package private) LogFile getLogFileForWrite​(LogBuffer lb)
      Called by LogBuffer.init() to obtain the LogFile that will be used to write a specific log block.
      (package private) java.lang.String getStats()
      Returns an XML node containing statistics for the LogFileManager.
      (package private) void init​(LogBufferManager bmgr)
      validate LogFiles and set member variables.
      (package private) long mark​(long key)
      calls mark(key, false)
      (package private) long mark​(long key, boolean force)
      sets the LogFile's mark.
      (package private) void open()
      open pool of LogFile(s).
      (package private) int read​(LogBuffer lb, int bsn)
      reads a block of data into LogBuffer lb .
      (package private) long setAutoMark​(boolean automark)
      Sets the LogFile marking mode.
      (package private) void setCurrentKey​(long key)
      updates currentKey member variable.
      private boolean setLockOnFile​(java.io.File name, boolean lock)
      Create a JVM wide lock on a File.
      (package private) void setLogEventListener​(LogEventListener eventListener)
      Registers a LogEventListener for log event notifications.
      (package private) void setMarkData​(java.nio.ByteBuffer data)
      generates MARKKEY data record into supplied data parameter.
      (package private) void validateFileHeader​(LogBuffer lb)
      Compares values in log file header record with current configuration.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • maxBlocksPerFile

        int maxBlocksPerFile
        maximum number of blocks to store in each LogFile.

        controls when logging is switched to a new log file, and/or when a circular log is reset to seek address zero.

        See Also:
        getLogFileForWrite(LogBuffer)

      • activeMark

        long activeMark
        The log key for the oldest active entry in the log.

        When automark is enabled (true) the activeMark is updated after every put() operation. When automark is disabled (as should be the case with JOTM) the activeMark is updated manually by a call to Logger.mark(), or during logOverflowNotification processing if a LogEventListener is registered.

        See Also:
        Logger.mark(long)

      • automark

        boolean automark
        indicates whether log files will be marked automatically.

        When automark is false, the mark() method must be invoked by the log user. When automark is true, the Logger will automatically set the mark at the most recent record.

      • restartAutoMark

        boolean restartAutoMark
        The automark value restored during log file initialization.

        if a set of log files exist from a prior execution, the value of automark is set based on the log file header record processed during the init() processing. The value restored from the log file is saved in restartAutoMark so it can be reported by getStats().

      • currentKey

        long currentKey
        last key returned by put().

        updated by setCurrentKey(long)

      • initialKey

        private long initialKey
        the first log key generated by this instance of the Logger.

        Initialized by init() to the block immediately following the last block written in the prior execution.

      • autoMarkOn

        final byte[] autoMarkOn
        data written to log when autoMark is turned on

      • autoMarkOff

        final byte[] autoMarkOff
        data written to log when autoMark is turned off

      • fileManagerLock

        private final java.lang.Object fileManagerLock
        lock controlling access to LogFile.

      • fileSet

        LogFile[] fileSet
        set of LogFile objects associated with the physical log files.
        See Also:
        open()

      • lfIndex

        int lfIndex
        workerID to current entry in fileSet[]

      • currentLogFile

        LogFile currentLogFile

      • fileHeader

        byte[][] fileHeader
        LogFile header record.

        protected by fileManagerLock

        The first record of every LogFile is a FILE_HEADER record containing information that is used during recovery to reposition the log file and replay records starting from the active mark. byte[1] autoMark byte[1] long activeMark byte[8] global to all log files long lowMark byte[8] low mark for current file == high mark for previous file long prevSwitchTod byte[8] time of previous file switch int fileSet.length byte[4] number of files in fileSet int maxBlocksPerFile byte[4] byte[2] crlf byte[2]

      • fileHeaderBB

        java.nio.ByteBuffer fileHeaderBB
        ByteBuffer wrapper for fileHeader to facilitate conversion of numeric information to byte[] format.

        protected by fileManagerLock

      • markRecord

        byte[][] markRecord
        MARK control Record.

        A MARK control record containing the current state of the automark mode and the active mark is stored at the beginning of every block. This strategy allows the logger to read through a log file looking only at the BSN for each block to locate the last written block. Once the last written block is located, the current state of the MARK can be obtained by examining the MARK record at the beginning of the block.

        To avokd garbage collection for a byte[] and ByteBuffer wrapper for each physical block, the class has a single markRecord member that is protected by fileManagerLock.

        protected by fileManagerLock byte[1] autoMark byte[1] long activeMark byte[8] global to all log files int fileSet.length byte[4] number of files in fileSet int maxBlocksPerFile byte[4] byte[2] crlf byte[2]

        See Also:
        setMarkData(ByteBuffer)

      • markRecordBB

        java.nio.ByteBuffer markRecordBB
        ByteBuffer wrapper for markRecord to facilitate conversion of numeric information to byte[] format.

        protected by fileManagerLock

      • crlf

        byte[] crlf
        end of line for log records to make logs readable in text editors.

      • bmgr

        LogBufferManager bmgr
        LogBufferManager used by methods that put() records to the log.

        For example, the mark() method uses bmgr to put() mark information to the log.

      • eventListener

        private LogEventListener eventListener
        The LogEventListener registered by the application that owns the logger. If eventListener is null then the application is not notified of log events.

      • eventManagerLock

        final java.lang.Object eventManagerLock
        Monitor used by EventManager thread to wait for events that need to be notified.

      • eventManagerThread

        java.lang.Thread eventManagerThread
        Thread used by LogFileManager to send events to the registered LogEventListener.

      • event

        int event
        event to be processed by the eventManagerThread.

        When a condition is encountered that requires the LogEventListener to be notified, the value is set to one of the event types.

      • LOG_OVERFLOW_EVENT

        static final int LOG_OVERFLOW_EVENT
        Event types for event manager thread.

        assign this value to the event member when invoking the event manager thread to handle a log overflow notification.

        See Also:
        Constant Field Values

      • lowestSafeLogKey

        private long lowestSafeLogKey
        The logKey to be used by LogEventListener.logOverflowNotification when moving records forward into the current log file.

      • initComplete

        private boolean initComplete
        Indicates that LogFileManager initialization is complete.

        Prior to being fully initialized, the LogFileManager should not attempt to put records to the journal because the buffer manager may not be initialized properly.

    • Constructor Detail

      • LogFileManager

        LogFileManager​(Configuration config)
        construct LogFileManager with Configuration supplied by caller.
        Parameters:
        config - Configuration object.

    • Method Detail

      • getLogFileForMark

        LogFile getLogFileForMark​(long mark)
        Returns the LogFile that contains the requested mark .

        Called by LogBufferManager to locate a log file needed for a replay() request.

        Parameters:
        mark - A log key previously returned by LogBufferManager.put(). The log key is used to compute the desired file.
        Returns:
        LogFile containing the requested mark .

        Returns null if none of the files in fileSet[] contain the requested mark.

      • getLogFileForWrite

        LogFile getLogFileForWrite​(LogBuffer lb)
                    throws LogFileOverflowException
        Called by LogBuffer.init() to obtain the LogFile that will be used to write a specific log block.

        The buffer sequence number of the LogBuffer parameter ( lf.bsn ) represents an implementation specific value that is used to manage log file space. As buffers are written to disk the buffer sequence number is incremented. The LogFileManager is able to compute the seek address for a buffer as a function of lf.bsn and buffer size when using LogBuffer implementations with fixed buffer sizes.

        In all cases, getLogFile records a header record into the buffer containing the current state of the automark mode and the current active mark.

        Parameters:
        lb - LogBuffer that is asking for the LogFile. LogFileManager implementations use lf.bsn to determine when to switch to a new file, or wrap a circular file back to seek address zero.
        Returns:
        a LogFile to use for writing the LogBuffer
        Throws:
        LogFileOverflowException

      • detectLogOverflow

        private void detectLogOverflow​(int bsn)
        Detect pending Log Overflow and notify event listener.

        if current file is 50% full then we check to see if the next file contains the active mark. We notify the event listener to move records forward to prevent log overflow.

        Called by: getLogFileForWrite(Logbuffer lb)

        Parameters:
        bsn - The block sequence number of the current LogBuffer.
        See Also:
        getLogFileForWrite(LogBuffer)

      • setMarkData

        void setMarkData​(java.nio.ByteBuffer data)
        generates MARKKEY data record into supplied data parameter.
        Parameters:
        data - ByteBuffer wrapping the target byte[]

      • mark

        long mark​(long key,
          boolean force)
   throws InvalidLogKeyException,
          java.io.IOException,
          java.lang.InterruptedException
        sets the LogFile's mark.

        mark() provides a generalized method for callers to inform the Logger that log space can be released for reuse.

        writes a MARKKEY control record to the log.

        Parameters:
        key - is an opaque log key returned by a previous call to put().
        force - when set true causes the caller to be blocked until the new force record is forced to disk.
        Returns:
        log key for the MARK record
        Throws:
        InvalidLogKeyException - if key parameter is out of range. key must be greater than current activeMark and less than the most recent key returned by put().
        java.io.IOException
        java.lang.InterruptedException

      • read

        int read​(LogBuffer lb,
         int bsn)
  throws java.io.IOException,
         InvalidLogBufferException
        reads a block of data into LogBuffer lb .

        Amount of data read is determined by lb.capacity().

        sets lb.lf with the fileSet[] entry that contains the requested BSN.

        Parameters:
        lb - LogBuffer to read data into.
        bsn - block sequence number of the block to be read. File position is computed as follows:
        1. locate the entry within fileSet[] that contains the requested bsn.
        2. compute position as (requested bsn - first bsn in file) * block size;
        Returns:
        block serial number of block read.

        returns -1 if the requested BSN does not exist in the current fileSet[].

        Throws:
        java.io.IOException
        InvalidLogBufferException

      • setAutoMark

        long setAutoMark​(boolean automark)
          throws InvalidLogKeyException,
                 java.io.IOException,
                 java.lang.InterruptedException,
                 LogFileOverflowException
        Sets the LogFile marking mode.

        writes an AUTOMARK control record to the log if the log is open.

        Parameters:
        automark - true to indicate automatic marking.
        Returns:
        log key for the generated MARK control record.
        Throws:
        InvalidLogKeyException
        java.io.IOException
        java.lang.InterruptedException
        LogFileOverflowException

      • setCurrentKey

        void setCurrentKey​(long key)
        updates currentKey member variable.

        Method must be synchronized to guarantee that only keys with larger values are assigned to currentKey.

        If automark mode is enabled, then the activeMark is updated as well.

        Parameters:
        key - a log key returned by the buffer manager.

      • getHighMark

        long getHighMark()
        Returns the highMark from the LogFile that is currently being written.

        Log keys greater than this value are beyond the logical end of the journal.

        Returns:
        highMark from the current LogFile.

      • setLogEventListener

        void setLogEventListener​(LogEventListener eventListener)
        Registers a LogEventListener for log event notifications.
        Parameters:
        eventListener - object to be notified of logger events.

      • setLockOnFile

        private boolean setLockOnFile​(java.io.File name,
                              boolean lock)
        Create a JVM wide lock on a File.

        Feature 30922

        The java.nio.channel.tryLock method is not guaranteed to respect locks that are set within the JVM by multiple threads. Consequently, it is possible for an application to create multiple instances of Logger resulting in corrupted log files as two separate instances of Logger write to the log.

        To prevent multiple instances of Logger from allocating the same files within a JVM, we create a system property

        Parameters:
        name - File object to be locked
        Returns:
        true if requested lock operation is successful.

      • init

        void init​(LogBufferManager bmgr)
   throws java.io.IOException,
          LogConfigurationException,
          InvalidLogBufferException,
          java.lang.InterruptedException
        validate LogFiles and set member variables.

        activeMark set based on last block written during previous execution.

        currentKey set to key of the last record written to the log.

        currentLogFile set to the next available LogFile in fileSet[] with file position set to resume writing at the next block following the last block written.

        Throws:
        java.io.IOException
        LogConfigurationException
        InvalidLogBufferException
        java.lang.InterruptedException

      • closeBufferManager

        void closeBufferManager()
                 throws java.lang.InterruptedException,
                        java.io.IOException
        Write a CLOSE record and shut down the buffer manager if we have one.

        This routine was originally inline in close(). It was refactored into a separate method to improve readability of close().

        Throws:
        java.lang.InterruptedException
        java.io.IOException

      • close

        void close()
    throws java.io.IOException,
           java.lang.InterruptedException
        Gracefully close the log files.
        Throws:
        java.io.IOException - If FileChannel.close() encounters an error.
        java.lang.InterruptedException
        See Also:
        AbstractInterruptibleChannel.close()

      • getStats

        java.lang.String getStats()
        Returns an XML node containing statistics for the LogFileManager.

        The nested element contain entries for each LogFile object in the set of log files.

        Returns:
        a String containing statistics as an XML node.