Package org.apache.jackrabbit.core.data
Class CachingDataStore
- java.lang.Object
- 
- org.apache.jackrabbit.core.data.AbstractDataStore
- 
- org.apache.jackrabbit.core.data.CachingDataStore
 
 
- 
- All Implemented Interfaces:
- AsyncTouchCallback,- AsyncUploadCallback,- DataStore,- MultiDataStoreAware
 - Direct Known Subclasses:
- CachingFDS,- S3DataStore,- VFSDataStore
 
 public abstract class CachingDataStore extends AbstractDataStore implements MultiDataStoreAware, AsyncUploadCallback, AsyncTouchCallback A caching data store that consists ofLocalCacheandBackend.Backendis single source of truth. All methods first try to fetch information fromLocalCache. If record is not available inLocalCache, then it is fetched fromBackendand saved toLocalCachefor further access. This class is designed to work withoutLocalCacheand then all information is fetched fromBackend. To disableLocalCachesetsetCacheSize(long)to 0. * Configuration:<DataStore class="org.apache.jackrabbit.aws.ext.ds.CachingDataStore"> <param name="path" value="/data/datastore"/> <param name="config" value="${rep.home}/backend.properties"/> <param name="cacheSize" value="68719476736"/> <param name="secret" value="123456"/> <param name="setCachePurgeTrigFactor(double)" value="0.95d"/> <param name="cacheSize" value="0.85d"/> <param name="minRecordLength" value="1024"/> <param name="continueOnAsyncUploadFailure" value="false"/> <param name="concurrentUploadsThreads" value="10"/> <param name="asyncUploadLimit" value="100"/> <param name="uploadRetries" value="3"/> <param name="touchAsync" value="false"/> <param name="proactiveCaching" value="true"/> <param name="recLengthCacheSize" value="200"/> </DataStore>
- 
- 
Field SummaryFields Modifier and Type Field Description protected Map<DataIdentifier,Long>asyncDownloadCacheIn memory map to hold in-progress asynchronous downloads.protected Map<DataIdentifier,Long>asyncTouchCacheIn memory map to hold in-progress asynchronous touch.protected Backendbackendprotected Map<DataIdentifier,WeakReference<DataIdentifier>>inUseAll data identifiers that are currently in use are in this set until they are garbage collected.protected Map<DataIdentifier,Long>recLenCacheIn memory cache to holdDataRecord.getLength()againstDataIdentifierprotected Map<DataIdentifier,Integer>uploadRetryMapIn memory map to hold failed asynchronous uploadDataIdentifierand its retry count.- 
Fields inherited from class org.apache.jackrabbit.core.data.AbstractDataStoreDIGEST
 
- 
 - 
Constructor SummaryConstructors Constructor Description CachingDataStore()
 - 
Method SummaryAll Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description DataRecordaddRecord(InputStream input)Creates a new data record inBackend.voidclearInUse()Clear the in-use list.voidclose()Close the data storebooleanconfirmDelete(DataIdentifier identifier)Method to confirm that identifier can be deleted fromBackendprotected abstract BackendcreateBackend()intdeleteAllOlderThan(long min)Delete objects that have a modified date older than the specified date.voiddeleteFromCache(DataIdentifier identifier)voiddeleteRecord(DataIdentifier identifier)This method deletes record fromBackendand then fromLocalCacheIterator<DataIdentifier>getAllIdentifiers()Retrieves all identifiers fromBackend.intgetAsyncUploadLimit()BackendgetBackend()doublegetCachePurgeResizeFactor()doublegetCachePurgeTrigFactor()longgetCacheSize()intgetConcurrentUploadsThreads()StringgetConfig()Return path of configuration properties.longgetLastModified(DataIdentifier identifier)longgetLength(DataIdentifier identifier)Return the length of record fromLocalCacheif available, otherwise retrieve it fromBackend.protected abstract StringgetMarkerFile()intgetMinRecordLength()Return mininum object length.protected byte[]getOrCreateReferenceKey()Returns the reference key of this data store.StringgetPath()Set<String>getPendingUploads()DataRecordgetRecord(DataIdentifier identifier)Returns the identified data record.DataRecordgetRecordIfStored(DataIdentifier identifier)Get a data record for the given identifier or null it data record doesn't exist inBackendintgetUploadRetries()voidinit(String homeDir)Initialized the data store.booleanisContinueOnAsyncUploadFailure()booleanisInUse(DataIdentifier identifier)voidonAbort(AsyncTouchResult result)Callback method for aborted asynchronous touch.voidonAbort(AsyncUploadResult result)Callback method for aborted asynchronous upload.voidonFailure(AsyncTouchResult result)Callback method for failed asynchronous touch.voidonFailure(AsyncUploadResult result)Callback method for failed asynchronous upload.voidonSuccess(AsyncTouchResult result)Callback method for successful asynchronous touch.voidonSuccess(AsyncUploadResult result)Callback method for successful asynchronous upload.voidsetAsyncUploadLimit(int asyncUploadLimit)voidsetCachePurgeResizeFactor(double cachePurgeResizeFactor)Set purge resize factor ofLocalCache.voidsetCachePurgeTrigFactor(double cachePurgeTrigFactor)Set purge trigger factor ofLocalCache.voidsetCacheSize(long cacheSize)Set size ofLocalCache.voidsetConcurrentUploadsThreads(int concurrentUploadsThreads)voidsetConfig(String config)Set the configuration properties path.voidsetContinueOnAsyncUploadFailure(boolean continueOnAsyncUploadFailure)voidsetMinRecordLength(int minRecordLength)Set the minimum object length.voidsetPath(String path)Set path ofLocalCache.voidsetProactiveCaching(boolean proactiveCaching)voidsetRecLengthCacheSize(int recLengthCacheSize)voidsetSecret(String secret)Setter for configuration based secretvoidsetTouchAsync(boolean touchAsync)voidsetUploadRetries(int uploadRetries)voidupdateModifiedDateOnAccess(long before)From now on, update the modified date of an object even when accessing it.- 
Methods inherited from class org.apache.jackrabbit.core.data.AbstractDataStoreencodeHexString, getRecordFromReference, getReferenceFromIdentifier
 
- 
 
- 
- 
- 
Field Detail- 
inUseprotected Map<DataIdentifier,WeakReference<DataIdentifier>> inUse All data identifiers that are currently in use are in this set until they are garbage collected.
 - 
uploadRetryMapprotected final Map<DataIdentifier,Integer> uploadRetryMap In memory map to hold failed asynchronous uploadDataIdentifierand its retry count. Once if all retries are exhausted or file is successfully uploaded, then corresponding entry is flushed from the map. As all failed uploads are synchronously uploaded at startup, this map is not required to be persisted.
 - 
asyncTouchCacheprotected final Map<DataIdentifier,Long> asyncTouchCache In memory map to hold in-progress asynchronous touch. Once touch is successful corresponding entry is flushed from the map.
 - 
asyncDownloadCacheprotected final Map<DataIdentifier,Long> asyncDownloadCache In memory map to hold in-progress asynchronous downloads. Once download is finished corresponding entry is flushed from the map.
 - 
recLenCacheprotected Map<DataIdentifier,Long> recLenCache In memory cache to holdDataRecord.getLength()againstDataIdentifier
 - 
backendprotected Backend backend 
 
- 
 - 
Method Detail- 
createBackendprotected abstract Backend createBackend() 
 - 
getMarkerFileprotected abstract String getMarkerFile() 
 - 
initpublic void init(String homeDir) throws RepositoryException Initialized the data store. If the path is not set, <repository home>/repository/datastore is used. This directory is automatically created if it does not yet exist. During first initialization, it upload all files from local datastore to backed and local datastore act as a local cache.- Specified by:
- initin interface- DataStore
- Parameters:
- homeDir- the home directory of the repository
- Throws:
- RepositoryException
 
 - 
addRecordpublic DataRecord addRecord(InputStream input) throws DataStoreException Creates a new data record inBackend. The stream is first consumed and the contents are saved in a temporary file and theAbstractDataStore.DIGESTmessage digest of the stream is calculated. If a record with the sameAbstractDataStore.DIGESTdigest (and length) is found then it is returned. Otherwise new record is created inBackendand the temporary file is moved in place toLocalCache.- Specified by:
- addRecordin interface- DataStore
- Parameters:
- input- binary stream
- Returns:
- CachingDataRecord
- Throws:
- DataStoreException- if the record could not be created.
 
 - 
getRecordpublic DataRecord getRecord(DataIdentifier identifier) throws DataStoreException Description copied from interface:DataStoreReturns the identified data record. The given identifier should be the identifier of a previously saved data record. Since records are never removed, there should never be cases where the identified record is not found. Abnormal cases like that are treated as errors and handled by throwing an exception.- Specified by:
- getRecordin interface- DataStore
- Overrides:
- getRecordin class- AbstractDataStore
- Parameters:
- identifier- data identifier
- Returns:
- identified data record
- Throws:
- DataStoreException- if the data store could not be accessed, or if the given identifier is invalid
 
 - 
getRecordIfStoredpublic DataRecord getRecordIfStored(DataIdentifier identifier) throws DataStoreException Get a data record for the given identifier or null it data record doesn't exist inBackend- Specified by:
- getRecordIfStoredin interface- DataStore
- Parameters:
- identifier- identifier of record.
- Returns:
- the CachingDataRecordor null.
- Throws:
- DataStoreException- if the data store could not be accessed
 
 - 
updateModifiedDateOnAccesspublic void updateModifiedDateOnAccess(long before) Description copied from interface:DataStoreFrom now on, update the modified date of an object even when accessing it. Usually, the modified date is only updated when creating a new object, or when a new link is added to an existing object. When this setting is enabled, even getLength() will update the modified date.- Specified by:
- updateModifiedDateOnAccessin interface- DataStore
- Parameters:
- before- - update the modified date to the current time if it is older than this value
 
 - 
getAllIdentifierspublic Iterator<DataIdentifier> getAllIdentifiers() throws DataStoreException Retrieves all identifiers fromBackend.- Specified by:
- getAllIdentifiersin interface- DataStore
- Returns:
- an iterator over all DataIdentifier objects
- Throws:
- DataStoreException- if the list could not be read
 
 - 
deleteRecordpublic void deleteRecord(DataIdentifier identifier) throws DataStoreException This method deletes record fromBackendand then fromLocalCache- Specified by:
- deleteRecordin interface- MultiDataStoreAware
- Parameters:
- identifier- data identifier
- Throws:
- DataStoreException- if the data store could not be accessed, or if the given identifier is invalid
 
 - 
deleteAllOlderThanpublic int deleteAllOlderThan(long min) throws DataStoreExceptionDescription copied from interface:DataStoreDelete objects that have a modified date older than the specified date.- Specified by:
- deleteAllOlderThanin interface- DataStore
- Parameters:
- min- the minimum time
- Returns:
- the number of data records deleted
- Throws:
- DataStoreException
 
 - 
getLastModifiedpublic long getLastModified(DataIdentifier identifier) throws DataStoreException - Throws:
- DataStoreException
 
 - 
getLengthpublic long getLength(DataIdentifier identifier) throws DataStoreException Return the length of record fromLocalCacheif available, otherwise retrieve it fromBackend.- Throws:
- DataStoreException
 
 - 
getOrCreateReferenceKeyprotected byte[] getOrCreateReferenceKey() throws DataStoreExceptionDescription copied from class:AbstractDataStoreReturns the reference key of this data store. If one does not already exist, it is automatically created in an implementation-specific way. The default implementation simply creates a temporary random key that's valid only until the data store gets restarted. Subclasses can override and/or decorate this method to support a more persistent reference key.This method is called only once during the lifetime of a data store instance and the return value is cached in memory, so it's no problem if the implementation is slow. - Overrides:
- getOrCreateReferenceKeyin class- AbstractDataStore
- Returns:
- reference key
- Throws:
- DataStoreException- if the key is not available
 
 - 
deleteFromCachepublic void deleteFromCache(DataIdentifier identifier) throws DataStoreException - Throws:
- DataStoreException
 
 - 
onSuccesspublic void onSuccess(AsyncUploadResult result) Description copied from interface:AsyncUploadCallbackCallback method for successful asynchronous upload.- Specified by:
- onSuccessin interface- AsyncUploadCallback
 
 - 
onFailurepublic void onFailure(AsyncUploadResult result) Description copied from interface:AsyncUploadCallbackCallback method for failed asynchronous upload.- Specified by:
- onFailurein interface- AsyncUploadCallback
 
 - 
onAbortpublic void onAbort(AsyncUploadResult result) Description copied from interface:AsyncUploadCallbackCallback method for aborted asynchronous upload.- Specified by:
- onAbortin interface- AsyncUploadCallback
 
 - 
onSuccesspublic void onSuccess(AsyncTouchResult result) Description copied from interface:AsyncTouchCallbackCallback method for successful asynchronous touch.- Specified by:
- onSuccessin interface- AsyncTouchCallback
 
 - 
onFailurepublic void onFailure(AsyncTouchResult result) Description copied from interface:AsyncTouchCallbackCallback method for failed asynchronous touch.- Specified by:
- onFailurein interface- AsyncTouchCallback
 
 - 
onAbortpublic void onAbort(AsyncTouchResult result) Description copied from interface:AsyncTouchCallbackCallback method for aborted asynchronous touch.- Specified by:
- onAbortin interface- AsyncTouchCallback
 
 - 
confirmDeletepublic boolean confirmDelete(DataIdentifier identifier) Method to confirm that identifier can be deleted fromBackend- Parameters:
- identifier-
- Returns:
 
 - 
clearInUsepublic void clearInUse() Description copied from interface:DataStoreClear the in-use list. This is only used for testing to make the the garbage collection think that objects are no longer in use.- Specified by:
- clearInUsein interface- DataStore
 
 - 
isInUsepublic boolean isInUse(DataIdentifier identifier) 
 - 
closepublic void close() throws DataStoreExceptionDescription copied from interface:DataStoreClose the data store- Specified by:
- closein interface- DataStore
- Throws:
- DataStoreException- if a problem occurred
 
 - 
setSecretpublic void setSecret(String secret) Setter for configuration based secret- Parameters:
- secret- the secret used to sign reference binaries
 
 - 
setMinRecordLengthpublic void setMinRecordLength(int minRecordLength) Set the minimum object length.- Parameters:
- minRecordLength- the length
 
 - 
getMinRecordLengthpublic int getMinRecordLength() Return mininum object length.- Specified by:
- getMinRecordLengthin interface- DataStore
- Returns:
- the minimum size in bytes
 
 - 
getConfigpublic String getConfig() Return path of configuration properties.- Returns:
- path of configuration properties.
 
 - 
setConfigpublic void setConfig(String config) Set the configuration properties path.- Parameters:
- config- path of configuration properties.
 
 - 
getCacheSizepublic long getCacheSize() - Returns:
- size of LocalCache.
 
 - 
setCacheSizepublic void setCacheSize(long cacheSize) Set size ofLocalCache.- Parameters:
- cacheSize- size of- LocalCache.
 
 - 
getPathpublic String getPath() - Returns:
- path of LocalCache.
 
 - 
setPathpublic void setPath(String path) Set path ofLocalCache.- Parameters:
- path- of- LocalCache.
 
 - 
getCachePurgeTrigFactorpublic double getCachePurgeTrigFactor() - Returns:
- Purge trigger factor of LocalCache.
 
 - 
setCachePurgeTrigFactorpublic void setCachePurgeTrigFactor(double cachePurgeTrigFactor) Set purge trigger factor ofLocalCache.- Parameters:
- cachePurgeTrigFactor- purge trigger factor.
 
 - 
getCachePurgeResizeFactorpublic double getCachePurgeResizeFactor() - Returns:
- Purge resize factor of LocalCache.
 
 - 
setCachePurgeResizeFactorpublic void setCachePurgeResizeFactor(double cachePurgeResizeFactor) Set purge resize factor ofLocalCache.- Parameters:
- cachePurgeResizeFactor- purge resize factor.
 
 - 
getConcurrentUploadsThreadspublic int getConcurrentUploadsThreads() 
 - 
setConcurrentUploadsThreadspublic void setConcurrentUploadsThreads(int concurrentUploadsThreads) 
 - 
getAsyncUploadLimitpublic int getAsyncUploadLimit() 
 - 
setAsyncUploadLimitpublic void setAsyncUploadLimit(int asyncUploadLimit) 
 - 
isContinueOnAsyncUploadFailurepublic boolean isContinueOnAsyncUploadFailure() 
 - 
setContinueOnAsyncUploadFailurepublic void setContinueOnAsyncUploadFailure(boolean continueOnAsyncUploadFailure) 
 - 
getUploadRetriespublic int getUploadRetries() 
 - 
setUploadRetriespublic void setUploadRetries(int uploadRetries) 
 - 
setTouchAsyncpublic void setTouchAsync(boolean touchAsync) 
 - 
setProactiveCachingpublic void setProactiveCaching(boolean proactiveCaching) 
 - 
setRecLengthCacheSizepublic void setRecLengthCacheSize(int recLengthCacheSize) 
 - 
getBackendpublic Backend getBackend() 
 
- 
 
-