Package loci.formats

Class Memoizer

java.lang.Object
loci.formats.ReaderWrapper
loci.formats.Memoizer
All Implemented Interfaces:
Closeable, AutoCloseable, ICompressedTileReader, IFormatHandler, IFormatReader, IMetadataConfigurable, IPyramidHandler
public class Memoizer extends ReaderWrapper
ReaderWrapper implementation which caches the state of the delegate (including and other ReaderWrapper instances) after setId(String) has been called. Initializing a Bio-Formats reader can consume substantial time and memory. Most of the initialization time is spent in the setId(String) call. Various factors can impact the performance of this step including the file size, the amount of metadata in the image and also the file format itself. With the Memoizer reader wrapper, if the time required to call the setId(String) method is larger than minimumElapsed, the initialized reader including all reader wrappers will be cached in a memo file via saveMemo(). Any subsequent call to setId(String) with a reader decorated by the Memoizer on the same input file will load the reader from the memo file using loadMemo() instead of performing a full reader initialization. In essence, the speed-up gained from memoization will happen only after the first initialization of the reader for a particular file.

  • Field Details

    • VERSION

      public static final Integer VERSION
      Default file version. Bumping this number will invalidate all other cached items. This should happen when the order and type of objects stored in the memo file changes.

    • DEFAULT_MINIMUM_ELAPSED

      public static final long DEFAULT_MINIMUM_ELAPSED
      Default value for minimumElapsed if none is provided in the constructor.
      See Also:

    • LOGGER

      private static final org.slf4j.Logger LOGGER
      Default Logger for the memoizer class

    • minimumElapsed

      private final long minimumElapsed
      Minimum number of milliseconds which must elapse during the call to setId(java.lang.String) before a memo file will be created. Default to DEFAULT_MINIMUM_ELAPSED if not specified via the constructor.

    • directory

      private final File directory
      Directory where all memo files should be created. If this value is non-null, then all memo files will be created under it. Can be overriden by doInPlaceCaching.

    • doInPlaceCaching

      private boolean doInPlaceCaching
      If true, then all memo files will be created in the same directory as the original file.

    • ser

      protected transient Memoizer.Deser ser

    • service

      private transient OMEXMLService service

    • realFile

      private loci.common.Location realFile

    • memoFile

      private File memoFile

    • tempFile

      private File tempFile

    • skipLoad

      private boolean skipLoad

    • skipSave

      private boolean skipSave

    • versionChecking

      private boolean versionChecking
      Boolean specifying whether to invalidate the memo file based upon mismatched major/minor version numbers. By default, the Git commit hash is used to invalidate the memo file.

    • loadedFromMemo

      private boolean loadedFromMemo
      Whether the ReaderWrapper.reader instance currently active was loaded from the memo file during setId(String).

    • savedToMemo

      private boolean savedToMemo
      Whether the ReaderWrapper.reader instance was saved to a memo file on setId(String).

    • userMetadataStore

      private MetadataStore userMetadataStore
      MetadataStore set by the caller. This value will be held locally and not set on the ReaderWrapper.reader delegate until the execution of setId(String). If no value has been set by the caller, then no special actions are taken during setId(String). If a value is set, however, we must be careful with attempting to serialize it
      See Also:

    • replacementMetadataStore

      private MetadataStore replacementMetadataStore
      MetadataStore created internally.
      See Also:

  • Constructor Details

    • Memoizer

      public Memoizer()
      Constructs a memoizer around a new ImageReader creating memo files under the same directory as the original file only if the call to setId(java.lang.String) takes longer than 100L in milliseconds.

    • Memoizer

      public Memoizer(long minimumElapsed)
      Constructs a memoizer around a new ImageReader creating memo files under the same directory as the original file only if the call to setId(java.lang.String) takes longer than minimumElapsed in milliseconds.
      Parameters:
      minimumElapsed - a long specifying the number of milliseconds which must elapse during the call to setId(java.lang.String) before a memo file will be created.

    • Memoizer

      public Memoizer(long minimumElapsed, File directory)
      Constructs a memoizer around a new ImageReader creating memo files under the directory argument including the full path of the original file only if the call to setId(java.lang.String) takes longer than minimumElapsed in milliseconds.
      Parameters:
      minimumElapsed - a long specifying the number of milliseconds which must elapse during the call to setId(java.lang.String) before a memo file will be created.
      directory - a File specifying the directory where all memo files should be created. If null, disable memoization.

    • Memoizer

      public Memoizer(IFormatReader r)
      Constructs a memoizer around the given IFormatReader creating memo files under the same directory as the original file only if the call to setId(java.lang.String) takes longer than 100L in milliseconds.
      Parameters:
      r - an IFormatReader instance. If null, a new ImageReader is created.

    • Memoizer

      public Memoizer(IFormatReader r, long minimumElapsed)
      Constructs a memoizer around the given IFormatReader creating memo files under the same directory as the original file only if the call to setId(java.lang.String) takes longer than minimumElapsed in milliseconds.
      Parameters:
      r - an IFormatReader instance If null, a new ImageReader is created.
      minimumElapsed - a long specifying the number of milliseconds which must elapse during the call to setId(java.lang.String) before a memo file will be created.

    • Memoizer

      public Memoizer(IFormatReader r, File directory)
      Constructs a memoizer around the given IFormatReader creating memo file under the directory argument including the full path of the original file only if the call to setId(java.lang.String) takes longer than 100L in milliseconds.
      Parameters:
      r - an IFormatReader instance If null, a new ImageReader is created.
      directory - a File specifying the directory where all memo files should be created. If null, disable memoization.

    • Memoizer

      public Memoizer(IFormatReader r, long minimumElapsed, File directory)
      Constructs a memoizer around the given IFormatReader creating memo files under the directory argument including the full path of the original file only if the call to setId(java.lang.String) takes longer than minimumElapsed in milliseconds.
      Parameters:
      r - an IFormatReader instance If null, a new ImageReader is created.
      minimumElapsed - a long specifying the number of milliseconds which must elapse during the call to setId(java.lang.String) before a memo file will be created.
      directory - a File specifying the directory where all memo files should be created. If null, disable memoization.

  • Method Details

    • isLoadedFromMemo

      public boolean isLoadedFromMemo()
      Returns whether the ReaderWrapper.reader instance currently active was loaded from the memo file during setId(String).
      Returns:
      true if the reader was loaded from the memo file, false otherwise.

    • isSavedToMemo

      public boolean isSavedToMemo()
      Returns whether the ReaderWrapper.reader instance currently active was saved to the memo file during setId(String).
      Returns:
      true if the reader was saved to the memo file, false otherwise.

    • isVersionChecking

      public boolean isVersionChecking()
      Returns whether or not version checking is done based upon major/minor version numbers.
      Returns:
      true if version checking is done based upon major/minor version numbers, false otherwise.

    • versionMismatch

      public boolean versionMismatch() throws IOException
      Returns true if the version of the memo file as returned by Memoizer.Deser.loadReleaseVersion() and Memoizer.Deser.loadRevision() do not match the current version as specified by FormatTools.VERSION.
      Throws:
      IOException

    • setVersionChecking

      public void setVersionChecking(boolean version)
      Set whether version checking is done based upon major/minor version numbers. If true, then a mismatch between the major/minor version of the calling code (e.g. 4.4) and the major/minor version saved in the memo file (e.g. 5.0) will result in the memo file being invalidated. If false (default), a mismatch in the Git commit hashes will invalidate the memo file. This method allows for less strict version checking.
      Parameters:
      version - a boolean specifying whether version checking is done based upon major/minor version numbers to invalidate the memo file

    • skipSave

      public void skipSave(boolean skip)
      Set whether a memo file should be saved if a valid file is not present. If false (default), then a memo file will be saved. This may cause an existing memo file to be overwritten. If true, then a memo file will not be saved. This effectively makes the current Memoizer read-only.

    • cleanup

      protected void cleanup()

    • close

      public void close() throws IOException
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Overrides:
      close in class ReaderWrapper
      Throws:
      IOException

    • close

      public void close(boolean fileOnly) throws IOException
      Description copied from interface: IFormatReader
      Closes the currently open file. If the flag is set, this is all that happens; if unset, it is equivalent to calling Closeable.close().
      Specified by:
      close in interface IFormatReader
      Overrides:
      close in class ReaderWrapper
      Throws:
      IOException

    • setId

      public void setId(String id) throws FormatException, IOException
      Description copied from interface: IFormatHandler
      Sets the current file name.
      Specified by:
      setId in interface IFormatHandler
      Overrides:
      setId in class ReaderWrapper
      Throws:
      FormatException
      IOException

    • setMetadataStore

      public void setMetadataStore(MetadataStore store)
      Description copied from interface: IFormatReader
      Sets the default metadata store for this reader.
      Specified by:
      setMetadataStore in interface IFormatReader
      Overrides:
      setMetadataStore in class ReaderWrapper
      Parameters:
      store - a metadata store implementation.

    • getMetadataStore

      public MetadataStore getMetadataStore()
      Description copied from interface: IFormatReader
      Retrieves the current metadata store for this reader. You can be assured that this method will never return a null metadata store.
      Specified by:
      getMetadataStore in interface IFormatReader
      Overrides:
      getMetadataStore in class ReaderWrapper
      Returns:
      A metadata store implementation.

    • deleteQuietly

      protected boolean deleteQuietly(File file)
      Attempts to delete an existing file, logging at warn if the deletion returns false or at error if an exception is thrown.
      Returns:
      the result from File.delete() or false if an exception is thrown.

    • getDeser

      protected Memoizer.Deser getDeser()
      Returns a configured Kryo instance. This method can be modified by consumers. The returned instance is not thread-safe.
      Returns:
      a non-null Kryo instance.

    • getService

      protected OMEXMLService getService() throws MissingLibraryException
      Throws:
      MissingLibraryException

    • stopWatch

      protected org.perf4j.slf4j.Slf4JStopWatch stopWatch()

    • getMemoFile

      public File getMemoFile(String id)
      Constructs a File object from id string. This method can be modified by consumers, but then existing memo files will not be found.
      Parameters:
      id - the path passed to setId(java.lang.String)
      Returns:
      a File object pointing at the location of the memo file

    • isWritableDirectory

      protected boolean isWritableDirectory(File writeDirectory)
      Test if the given File instance is a writable directory.
      Parameters:
      writeDirectory - a possible writable directory
      Returns:
      if the given File is indeed a writable directory

    • loadMemo

      public IFormatReader loadMemo() throws IOException, FormatException
      Load a memo file if possible, returning a null if not. Corrupt memo files will be deleted if possible. Kryo exceptions should never propagate to the caller. Only the regular Bio-Formats exceptions should be thrown.
      Throws:
      IOException
      FormatException

    • saveMemo

      public boolean saveMemo()
      Save a reader including all reader wrappers inside a memo file.

    • deleteMemo

      public boolean deleteMemo()
      Force the current memo file to be deleted, if it exists.
      Returns:
      true if the delete succeeded

    • getMemoFile

      public File getMemoFile()
      Returns:
      current memo file (may be null)

    • handleMetadataStore

      protected IFormatReader handleMetadataStore(IFormatReader memo) throws MissingLibraryException
      Return the IFormatReader instance that is passed in or null if it has been invalidated, which will include the instance being closed.
      Throws:
      MissingLibraryException

    • generateMemo

      public boolean generateMemo(String file) throws IOException
      Convenience method to generate (or regenerate) the memo file for a given file. Uses the cache directory and timing settings passed via Memoizer's constructor.
      Parameters:
      file - the file for which to generate a memo file
      Returns:
      true if a memo file was saved
      Throws:
      IOException

    • main

      public static void main(String[] args) throws Exception
      Throws:
      Exception

    • load

      private static void load(String id, File tmp, boolean delete) throws Exception
      Throws:
      Exception