Class CachingDataStore

    • Field Detail

      • uploadRetryMap

        protected final Map<DataIdentifier,​Integer> uploadRetryMap
        In memory map to hold failed asynchronous upload DataIdentifier and its retry count. Once if all retries are exhausted or file is successfully uploaded, then corresponding entry is flushed from the map. As all failed uploads are synchronously uploaded at startup, this map is not required to be persisted.
      • asyncTouchCache

        protected final Map<DataIdentifier,​Long> asyncTouchCache
        In memory map to hold in-progress asynchronous touch. Once touch is successful corresponding entry is flushed from the map.
      • asyncDownloadCache

        protected final Map<DataIdentifier,​Long> asyncDownloadCache
        In memory map to hold in-progress asynchronous downloads. Once download is finished corresponding entry is flushed from the map.
      • backend

        protected Backend backend
    • Constructor Detail

      • CachingDataStore

        public CachingDataStore()
    • Method Detail

      • createBackend

        protected abstract Backend createBackend()
      • getMarkerFile

        protected abstract String getMarkerFile()
      • init

        public void init​(String homeDir)
                  throws RepositoryException
        Initialized the data store. If the path is not set, <repository home>/repository/datastore is used. This directory is automatically created if it does not yet exist. During first initialization, it upload all files from local datastore to backed and local datastore act as a local cache.
        Specified by:
        init in interface DataStore
        Parameters:
        homeDir - the home directory of the repository
        Throws:
        RepositoryException
      • getRecord

        public DataRecord getRecord​(DataIdentifier identifier)
                             throws DataStoreException
        Description copied from interface: DataStore
        Returns the identified data record. The given identifier should be the identifier of a previously saved data record. Since records are never removed, there should never be cases where the identified record is not found. Abnormal cases like that are treated as errors and handled by throwing an exception.
        Specified by:
        getRecord in interface DataStore
        Overrides:
        getRecord in class AbstractDataStore
        Parameters:
        identifier - data identifier
        Returns:
        identified data record
        Throws:
        DataStoreException - if the data store could not be accessed, or if the given identifier is invalid
      • updateModifiedDateOnAccess

        public void updateModifiedDateOnAccess​(long before)
        Description copied from interface: DataStore
        From now on, update the modified date of an object even when accessing it. Usually, the modified date is only updated when creating a new object, or when a new link is added to an existing object. When this setting is enabled, even getLength() will update the modified date.
        Specified by:
        updateModifiedDateOnAccess in interface DataStore
        Parameters:
        before - - update the modified date to the current time if it is older than this value
      • deleteAllOlderThan

        public int deleteAllOlderThan​(long min)
                               throws DataStoreException
        Description copied from interface: DataStore
        Delete objects that have a modified date older than the specified date.
        Specified by:
        deleteAllOlderThan in interface DataStore
        Parameters:
        min - the minimum time
        Returns:
        the number of data records deleted
        Throws:
        DataStoreException
      • getOrCreateReferenceKey

        protected byte[] getOrCreateReferenceKey()
                                          throws DataStoreException
        Description copied from class: AbstractDataStore
        Returns the reference key of this data store. If one does not already exist, it is automatically created in an implementation-specific way. The default implementation simply creates a temporary random key that's valid only until the data store gets restarted. Subclasses can override and/or decorate this method to support a more persistent reference key.

        This method is called only once during the lifetime of a data store instance and the return value is cached in memory, so it's no problem if the implementation is slow.

        Overrides:
        getOrCreateReferenceKey in class AbstractDataStore
        Returns:
        reference key
        Throws:
        DataStoreException - if the key is not available
      • getPendingUploads

        public Set<String> getPendingUploads()
      • confirmDelete

        public boolean confirmDelete​(DataIdentifier identifier)
        Method to confirm that identifier can be deleted from Backend
        Parameters:
        identifier -
        Returns:
      • clearInUse

        public void clearInUse()
        Description copied from interface: DataStore
        Clear the in-use list. This is only used for testing to make the the garbage collection think that objects are no longer in use.
        Specified by:
        clearInUse in interface DataStore
      • setSecret

        public void setSecret​(String secret)
        Setter for configuration based secret
        Parameters:
        secret - the secret used to sign reference binaries
      • setMinRecordLength

        public void setMinRecordLength​(int minRecordLength)
        Set the minimum object length.
        Parameters:
        minRecordLength - the length
      • getMinRecordLength

        public int getMinRecordLength()
        Return mininum object length.
        Specified by:
        getMinRecordLength in interface DataStore
        Returns:
        the minimum size in bytes
      • getConfig

        public String getConfig()
        Return path of configuration properties.
        Returns:
        path of configuration properties.
      • setConfig

        public void setConfig​(String config)
        Set the configuration properties path.
        Parameters:
        config - path of configuration properties.
      • getCacheSize

        public long getCacheSize()
        Returns:
        size of LocalCache.
      • setCacheSize

        public void setCacheSize​(long cacheSize)
        Set size of LocalCache.
        Parameters:
        cacheSize - size of LocalCache.
      • getCachePurgeTrigFactor

        public double getCachePurgeTrigFactor()
        Returns:
        Purge trigger factor of LocalCache.
      • setCachePurgeTrigFactor

        public void setCachePurgeTrigFactor​(double cachePurgeTrigFactor)
        Set purge trigger factor of LocalCache.
        Parameters:
        cachePurgeTrigFactor - purge trigger factor.
      • getCachePurgeResizeFactor

        public double getCachePurgeResizeFactor()
        Returns:
        Purge resize factor of LocalCache.
      • setCachePurgeResizeFactor

        public void setCachePurgeResizeFactor​(double cachePurgeResizeFactor)
        Set purge resize factor of LocalCache.
        Parameters:
        cachePurgeResizeFactor - purge resize factor.
      • getConcurrentUploadsThreads

        public int getConcurrentUploadsThreads()
      • setConcurrentUploadsThreads

        public void setConcurrentUploadsThreads​(int concurrentUploadsThreads)
      • getAsyncUploadLimit

        public int getAsyncUploadLimit()
      • setAsyncUploadLimit

        public void setAsyncUploadLimit​(int asyncUploadLimit)
      • isContinueOnAsyncUploadFailure

        public boolean isContinueOnAsyncUploadFailure()
      • setContinueOnAsyncUploadFailure

        public void setContinueOnAsyncUploadFailure​(boolean continueOnAsyncUploadFailure)
      • getUploadRetries

        public int getUploadRetries()
      • setUploadRetries

        public void setUploadRetries​(int uploadRetries)
      • setTouchAsync

        public void setTouchAsync​(boolean touchAsync)
      • setProactiveCaching

        public void setProactiveCaching​(boolean proactiveCaching)
      • setRecLengthCacheSize

        public void setRecLengthCacheSize​(int recLengthCacheSize)
      • getBackend

        public Backend getBackend()