Package org.apache.jackrabbit.core.data
Class CachingDataStore
- java.lang.Object
-
- org.apache.jackrabbit.core.data.AbstractDataStore
-
- org.apache.jackrabbit.core.data.CachingDataStore
-
- All Implemented Interfaces:
AsyncTouchCallback,AsyncUploadCallback,DataStore,MultiDataStoreAware
- Direct Known Subclasses:
CachingFDS,S3DataStore,VFSDataStore
public abstract class CachingDataStore extends AbstractDataStore implements MultiDataStoreAware, AsyncUploadCallback, AsyncTouchCallback
A caching data store that consists ofLocalCacheandBackend.Backendis single source of truth. All methods first try to fetch information fromLocalCache. If record is not available inLocalCache, then it is fetched fromBackendand saved toLocalCachefor further access. This class is designed to work withoutLocalCacheand then all information is fetched fromBackend. To disableLocalCachesetsetCacheSize(long)to 0. * Configuration:<DataStore class="org.apache.jackrabbit.aws.ext.ds.CachingDataStore"> <param name="path" value="/data/datastore"/> <param name="config" value="${rep.home}/backend.properties"/> <param name="cacheSize" value="68719476736"/> <param name="secret" value="123456"/> <param name="setCachePurgeTrigFactor(double)" value="0.95d"/> <param name="cacheSize" value="0.85d"/> <param name="minRecordLength" value="1024"/> <param name="continueOnAsyncUploadFailure" value="false"/> <param name="concurrentUploadsThreads" value="10"/> <param name="asyncUploadLimit" value="100"/> <param name="uploadRetries" value="3"/> <param name="touchAsync" value="false"/> <param name="proactiveCaching" value="true"/> <param name="recLengthCacheSize" value="200"/> </DataStore>
-
-
Field Summary
Fields Modifier and Type Field Description protected Map<DataIdentifier,Long>asyncDownloadCacheIn memory map to hold in-progress asynchronous downloads.protected Map<DataIdentifier,Long>asyncTouchCacheIn memory map to hold in-progress asynchronous touch.protected Backendbackendprotected Map<DataIdentifier,WeakReference<DataIdentifier>>inUseAll data identifiers that are currently in use are in this set until they are garbage collected.protected Map<DataIdentifier,Long>recLenCacheIn memory cache to holdDataRecord.getLength()againstDataIdentifierprotected Map<DataIdentifier,Integer>uploadRetryMapIn memory map to hold failed asynchronous uploadDataIdentifierand its retry count.-
Fields inherited from class org.apache.jackrabbit.core.data.AbstractDataStore
DIGEST
-
-
Constructor Summary
Constructors Constructor Description CachingDataStore()
-
Method Summary
All Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description DataRecordaddRecord(InputStream input)Creates a new data record inBackend.voidclearInUse()Clear the in-use list.voidclose()Close the data storebooleanconfirmDelete(DataIdentifier identifier)Method to confirm that identifier can be deleted fromBackendprotected abstract BackendcreateBackend()intdeleteAllOlderThan(long min)Delete objects that have a modified date older than the specified date.voiddeleteFromCache(DataIdentifier identifier)voiddeleteRecord(DataIdentifier identifier)This method deletes record fromBackendand then fromLocalCacheIterator<DataIdentifier>getAllIdentifiers()Retrieves all identifiers fromBackend.intgetAsyncUploadLimit()BackendgetBackend()doublegetCachePurgeResizeFactor()doublegetCachePurgeTrigFactor()longgetCacheSize()intgetConcurrentUploadsThreads()StringgetConfig()Return path of configuration properties.longgetLastModified(DataIdentifier identifier)longgetLength(DataIdentifier identifier)Return the length of record fromLocalCacheif available, otherwise retrieve it fromBackend.protected abstract StringgetMarkerFile()intgetMinRecordLength()Return mininum object length.protected byte[]getOrCreateReferenceKey()Returns the reference key of this data store.StringgetPath()Set<String>getPendingUploads()DataRecordgetRecord(DataIdentifier identifier)Returns the identified data record.DataRecordgetRecordIfStored(DataIdentifier identifier)Get a data record for the given identifier or null it data record doesn't exist inBackendintgetUploadRetries()voidinit(String homeDir)Initialized the data store.booleanisContinueOnAsyncUploadFailure()booleanisInUse(DataIdentifier identifier)voidonAbort(AsyncTouchResult result)Callback method for aborted asynchronous touch.voidonAbort(AsyncUploadResult result)Callback method for aborted asynchronous upload.voidonFailure(AsyncTouchResult result)Callback method for failed asynchronous touch.voidonFailure(AsyncUploadResult result)Callback method for failed asynchronous upload.voidonSuccess(AsyncTouchResult result)Callback method for successful asynchronous touch.voidonSuccess(AsyncUploadResult result)Callback method for successful asynchronous upload.voidsetAsyncUploadLimit(int asyncUploadLimit)voidsetCachePurgeResizeFactor(double cachePurgeResizeFactor)Set purge resize factor ofLocalCache.voidsetCachePurgeTrigFactor(double cachePurgeTrigFactor)Set purge trigger factor ofLocalCache.voidsetCacheSize(long cacheSize)Set size ofLocalCache.voidsetConcurrentUploadsThreads(int concurrentUploadsThreads)voidsetConfig(String config)Set the configuration properties path.voidsetContinueOnAsyncUploadFailure(boolean continueOnAsyncUploadFailure)voidsetMinRecordLength(int minRecordLength)Set the minimum object length.voidsetPath(String path)Set path ofLocalCache.voidsetProactiveCaching(boolean proactiveCaching)voidsetRecLengthCacheSize(int recLengthCacheSize)voidsetSecret(String secret)Setter for configuration based secretvoidsetTouchAsync(boolean touchAsync)voidsetUploadRetries(int uploadRetries)voidupdateModifiedDateOnAccess(long before)From now on, update the modified date of an object even when accessing it.-
Methods inherited from class org.apache.jackrabbit.core.data.AbstractDataStore
encodeHexString, getRecordFromReference, getReferenceFromIdentifier
-
-
-
-
Field Detail
-
inUse
protected Map<DataIdentifier,WeakReference<DataIdentifier>> inUse
All data identifiers that are currently in use are in this set until they are garbage collected.
-
uploadRetryMap
protected final Map<DataIdentifier,Integer> uploadRetryMap
In memory map to hold failed asynchronous uploadDataIdentifierand its retry count. Once if all retries are exhausted or file is successfully uploaded, then corresponding entry is flushed from the map. As all failed uploads are synchronously uploaded at startup, this map is not required to be persisted.
-
asyncTouchCache
protected final Map<DataIdentifier,Long> asyncTouchCache
In memory map to hold in-progress asynchronous touch. Once touch is successful corresponding entry is flushed from the map.
-
asyncDownloadCache
protected final Map<DataIdentifier,Long> asyncDownloadCache
In memory map to hold in-progress asynchronous downloads. Once download is finished corresponding entry is flushed from the map.
-
recLenCache
protected Map<DataIdentifier,Long> recLenCache
In memory cache to holdDataRecord.getLength()againstDataIdentifier
-
backend
protected Backend backend
-
-
Method Detail
-
createBackend
protected abstract Backend createBackend()
-
getMarkerFile
protected abstract String getMarkerFile()
-
init
public void init(String homeDir) throws RepositoryException
Initialized the data store. If the path is not set, <repository home>/repository/datastore is used. This directory is automatically created if it does not yet exist. During first initialization, it upload all files from local datastore to backed and local datastore act as a local cache.- Specified by:
initin interfaceDataStore- Parameters:
homeDir- the home directory of the repository- Throws:
RepositoryException
-
addRecord
public DataRecord addRecord(InputStream input) throws DataStoreException
Creates a new data record inBackend. The stream is first consumed and the contents are saved in a temporary file and theAbstractDataStore.DIGESTmessage digest of the stream is calculated. If a record with the sameAbstractDataStore.DIGESTdigest (and length) is found then it is returned. Otherwise new record is created inBackendand the temporary file is moved in place toLocalCache.- Specified by:
addRecordin interfaceDataStore- Parameters:
input- binary stream- Returns:
CachingDataRecord- Throws:
DataStoreException- if the record could not be created.
-
getRecord
public DataRecord getRecord(DataIdentifier identifier) throws DataStoreException
Description copied from interface:DataStoreReturns the identified data record. The given identifier should be the identifier of a previously saved data record. Since records are never removed, there should never be cases where the identified record is not found. Abnormal cases like that are treated as errors and handled by throwing an exception.- Specified by:
getRecordin interfaceDataStore- Overrides:
getRecordin classAbstractDataStore- Parameters:
identifier- data identifier- Returns:
- identified data record
- Throws:
DataStoreException- if the data store could not be accessed, or if the given identifier is invalid
-
getRecordIfStored
public DataRecord getRecordIfStored(DataIdentifier identifier) throws DataStoreException
Get a data record for the given identifier or null it data record doesn't exist inBackend- Specified by:
getRecordIfStoredin interfaceDataStore- Parameters:
identifier- identifier of record.- Returns:
- the
CachingDataRecordor null. - Throws:
DataStoreException- if the data store could not be accessed
-
updateModifiedDateOnAccess
public void updateModifiedDateOnAccess(long before)
Description copied from interface:DataStoreFrom now on, update the modified date of an object even when accessing it. Usually, the modified date is only updated when creating a new object, or when a new link is added to an existing object. When this setting is enabled, even getLength() will update the modified date.- Specified by:
updateModifiedDateOnAccessin interfaceDataStore- Parameters:
before- - update the modified date to the current time if it is older than this value
-
getAllIdentifiers
public Iterator<DataIdentifier> getAllIdentifiers() throws DataStoreException
Retrieves all identifiers fromBackend.- Specified by:
getAllIdentifiersin interfaceDataStore- Returns:
- an iterator over all DataIdentifier objects
- Throws:
DataStoreException- if the list could not be read
-
deleteRecord
public void deleteRecord(DataIdentifier identifier) throws DataStoreException
This method deletes record fromBackendand then fromLocalCache- Specified by:
deleteRecordin interfaceMultiDataStoreAware- Parameters:
identifier- data identifier- Throws:
DataStoreException- if the data store could not be accessed, or if the given identifier is invalid
-
deleteAllOlderThan
public int deleteAllOlderThan(long min) throws DataStoreExceptionDescription copied from interface:DataStoreDelete objects that have a modified date older than the specified date.- Specified by:
deleteAllOlderThanin interfaceDataStore- Parameters:
min- the minimum time- Returns:
- the number of data records deleted
- Throws:
DataStoreException
-
getLastModified
public long getLastModified(DataIdentifier identifier) throws DataStoreException
- Throws:
DataStoreException
-
getLength
public long getLength(DataIdentifier identifier) throws DataStoreException
Return the length of record fromLocalCacheif available, otherwise retrieve it fromBackend.- Throws:
DataStoreException
-
getOrCreateReferenceKey
protected byte[] getOrCreateReferenceKey() throws DataStoreExceptionDescription copied from class:AbstractDataStoreReturns the reference key of this data store. If one does not already exist, it is automatically created in an implementation-specific way. The default implementation simply creates a temporary random key that's valid only until the data store gets restarted. Subclasses can override and/or decorate this method to support a more persistent reference key.This method is called only once during the lifetime of a data store instance and the return value is cached in memory, so it's no problem if the implementation is slow.
- Overrides:
getOrCreateReferenceKeyin classAbstractDataStore- Returns:
- reference key
- Throws:
DataStoreException- if the key is not available
-
deleteFromCache
public void deleteFromCache(DataIdentifier identifier) throws DataStoreException
- Throws:
DataStoreException
-
onSuccess
public void onSuccess(AsyncUploadResult result)
Description copied from interface:AsyncUploadCallbackCallback method for successful asynchronous upload.- Specified by:
onSuccessin interfaceAsyncUploadCallback
-
onFailure
public void onFailure(AsyncUploadResult result)
Description copied from interface:AsyncUploadCallbackCallback method for failed asynchronous upload.- Specified by:
onFailurein interfaceAsyncUploadCallback
-
onAbort
public void onAbort(AsyncUploadResult result)
Description copied from interface:AsyncUploadCallbackCallback method for aborted asynchronous upload.- Specified by:
onAbortin interfaceAsyncUploadCallback
-
onSuccess
public void onSuccess(AsyncTouchResult result)
Description copied from interface:AsyncTouchCallbackCallback method for successful asynchronous touch.- Specified by:
onSuccessin interfaceAsyncTouchCallback
-
onFailure
public void onFailure(AsyncTouchResult result)
Description copied from interface:AsyncTouchCallbackCallback method for failed asynchronous touch.- Specified by:
onFailurein interfaceAsyncTouchCallback
-
onAbort
public void onAbort(AsyncTouchResult result)
Description copied from interface:AsyncTouchCallbackCallback method for aborted asynchronous touch.- Specified by:
onAbortin interfaceAsyncTouchCallback
-
confirmDelete
public boolean confirmDelete(DataIdentifier identifier)
Method to confirm that identifier can be deleted fromBackend- Parameters:
identifier-- Returns:
-
clearInUse
public void clearInUse()
Description copied from interface:DataStoreClear the in-use list. This is only used for testing to make the the garbage collection think that objects are no longer in use.- Specified by:
clearInUsein interfaceDataStore
-
isInUse
public boolean isInUse(DataIdentifier identifier)
-
close
public void close() throws DataStoreExceptionDescription copied from interface:DataStoreClose the data store- Specified by:
closein interfaceDataStore- Throws:
DataStoreException- if a problem occurred
-
setSecret
public void setSecret(String secret)
Setter for configuration based secret- Parameters:
secret- the secret used to sign reference binaries
-
setMinRecordLength
public void setMinRecordLength(int minRecordLength)
Set the minimum object length.- Parameters:
minRecordLength- the length
-
getMinRecordLength
public int getMinRecordLength()
Return mininum object length.- Specified by:
getMinRecordLengthin interfaceDataStore- Returns:
- the minimum size in bytes
-
getConfig
public String getConfig()
Return path of configuration properties.- Returns:
- path of configuration properties.
-
setConfig
public void setConfig(String config)
Set the configuration properties path.- Parameters:
config- path of configuration properties.
-
getCacheSize
public long getCacheSize()
- Returns:
- size of
LocalCache.
-
setCacheSize
public void setCacheSize(long cacheSize)
Set size ofLocalCache.- Parameters:
cacheSize- size ofLocalCache.
-
getPath
public String getPath()
- Returns:
- path of
LocalCache.
-
setPath
public void setPath(String path)
Set path ofLocalCache.- Parameters:
path- ofLocalCache.
-
getCachePurgeTrigFactor
public double getCachePurgeTrigFactor()
- Returns:
- Purge trigger factor of
LocalCache.
-
setCachePurgeTrigFactor
public void setCachePurgeTrigFactor(double cachePurgeTrigFactor)
Set purge trigger factor ofLocalCache.- Parameters:
cachePurgeTrigFactor- purge trigger factor.
-
getCachePurgeResizeFactor
public double getCachePurgeResizeFactor()
- Returns:
- Purge resize factor of
LocalCache.
-
setCachePurgeResizeFactor
public void setCachePurgeResizeFactor(double cachePurgeResizeFactor)
Set purge resize factor ofLocalCache.- Parameters:
cachePurgeResizeFactor- purge resize factor.
-
getConcurrentUploadsThreads
public int getConcurrentUploadsThreads()
-
setConcurrentUploadsThreads
public void setConcurrentUploadsThreads(int concurrentUploadsThreads)
-
getAsyncUploadLimit
public int getAsyncUploadLimit()
-
setAsyncUploadLimit
public void setAsyncUploadLimit(int asyncUploadLimit)
-
isContinueOnAsyncUploadFailure
public boolean isContinueOnAsyncUploadFailure()
-
setContinueOnAsyncUploadFailure
public void setContinueOnAsyncUploadFailure(boolean continueOnAsyncUploadFailure)
-
getUploadRetries
public int getUploadRetries()
-
setUploadRetries
public void setUploadRetries(int uploadRetries)
-
setTouchAsync
public void setTouchAsync(boolean touchAsync)
-
setProactiveCaching
public void setProactiveCaching(boolean proactiveCaching)
-
setRecLengthCacheSize
public void setRecLengthCacheSize(int recLengthCacheSize)
-
getBackend
public Backend getBackend()
-
-