org.apache.jackrabbit.core.data

Class CachingDataStore

java.lang.Object

org.apache.jackrabbit.core.data.AbstractDataStore

org.apache.jackrabbit.core.data.CachingDataStore

All Implemented Interfaces:

AsyncTouchCallback, AsyncUploadCallback, DataStore, MultiDataStoreAware

Direct Known Subclasses:

CachingFDS, S3DataStore

public abstract class CachingDataStore
extends AbstractDataStore
implements MultiDataStoreAware, AsyncUploadCallback, AsyncTouchCallback

A caching data store that consists of LocalCache and Backend. Backend is single source of truth. All methods first try to fetch information from LocalCache. If record is not available in LocalCache, then it is fetched from Backend and saved to LocalCache for further access. This class is designed to work without LocalCache and then all information is fetched from Backend. To disable LocalCache set setCacheSize(long) to 0. * Configuration:

 <DataStore class="org.apache.jackrabbit.aws.ext.ds.CachingDataStore">
 
     <param name="path" value="/data/datastore"/>
     <param name="config" value="${rep.home}/backend.properties"/>
     <param name="cacheSize" value="68719476736"/>
     <param name="secret" value="123456"/>
     <param name="setCachePurgeTrigFactor(double)" value="0.95d"/>
     <param name="cacheSize" value="0.85d"/>
     <param name="minRecordLength" value="1024"/>
     <param name="continueOnAsyncUploadFailure" value="false"/>
     <param name="concurrentUploadsThreads" value="10"/>
     <param name="asyncUploadLimit" value="100"/>
     <param name="uploadRetries" value="3"/>
     <param name="touchAsync" value="false"/>
     <param name="proactiveCaching" value="true"/>
     <param name="recLengthCacheSize" value="200"/>
 </DataStore>
 

Field Summary

Fields

Modifier and Type	Field and Description
protected Map<DataIdentifier,Long>	asyncDownloadCache In memory map to hold in-progress asynchronous downloads.
protected Map<DataIdentifier,Long>	asyncTouchCache In memory map to hold in-progress asynchronous touch.
protected Backend	backend
protected Map<DataIdentifier,WeakReference<DataIdentifier>>	inUse All data identifiers that are currently in use are in this set until they are garbage collected.
protected Map<DataIdentifier,Long>	recLenCache In memory cache to hold DataRecord.getLength() against DataIdentifier
protected Map<DataIdentifier,Integer>	uploadRetryMap In memory map to hold failed asynchronous upload DataIdentifier and its retry count.

Constructor Summary

Constructors

Constructor and Description
CachingDataStore()

Method Summary

Methods

Modifier and Type	Method and Description
DataRecord	addRecord(InputStream input) Creates a new data record in Backend.
void	clearInUse() Clear the in-use list.
void	close() Close the data store
boolean	confirmDelete(DataIdentifier identifier) Method to confirm that identifier can be deleted from Backend
protected abstract Backend	createBackend()
int	deleteAllOlderThan(long min) Delete objects that have a modified date older than the specified date.
void	deleteFromCache(DataIdentifier identifier)
void	deleteRecord(DataIdentifier identifier) This method deletes record from Backend and then from LocalCache
Iterator<DataIdentifier>	getAllIdentifiers() Retrieves all identifiers from Backend.
int	getAsyncUploadLimit()
Backend	getBackend()
double	getCachePurgeResizeFactor()
double	getCachePurgeTrigFactor()
long	getCacheSize()
int	getConcurrentUploadsThreads()
String	getConfig() Return path of configuration properties.
long	getLastModified(DataIdentifier identifier) Return lastModified of record from Backend assuming Backend as a single source of truth.
long	getLength(DataIdentifier identifier) Return the length of record from LocalCache if available, otherwise retrieve it from Backend.
protected abstract String	getMarkerFile()
int	getMinRecordLength() Return mininum object length.
protected byte[]	getOrCreateReferenceKey() Returns the reference key of this data store.
String	getPath()
Set<String>	getPendingUploads()
DataRecord	getRecord(DataIdentifier identifier) Returns the identified data record.
DataRecord	getRecordIfStored(DataIdentifier identifier) Get a data record for the given identifier or null it data record doesn't exist in Backend
int	getUploadRetries()
void	init(String homeDir) Initialized the data store.
boolean	isContinueOnAsyncUploadFailure()
boolean	isInUse(DataIdentifier identifier)
void	onAbort(AsyncTouchResult result) Callback method for aborted asynchronous touch.
void	onAbort(AsyncUploadResult result) Callback method for aborted asynchronous upload.
void	onFailure(AsyncTouchResult result) Callback method for failed asynchronous touch.
void	onFailure(AsyncUploadResult result) Callback method for failed asynchronous upload.
void	onSuccess(AsyncTouchResult result) Callback method for successful asynchronous touch.
void	onSuccess(AsyncUploadResult result) Callback method for successful asynchronous upload.
void	setAsyncUploadLimit(int asyncUploadLimit)
void	setCachePurgeResizeFactor(double cachePurgeResizeFactor) Set purge resize factor of LocalCache.
void	setCachePurgeTrigFactor(double cachePurgeTrigFactor) Set purge trigger factor of LocalCache.
void	setCacheSize(long cacheSize) Set size of LocalCache.
void	setConcurrentUploadsThreads(int concurrentUploadsThreads)
void	setConfig(String config) Set the configuration properties path.
void	setContinueOnAsyncUploadFailure(boolean continueOnAsyncUploadFailure)
void	setMinRecordLength(int minRecordLength) Set the minimum object length.
void	setPath(String path) Set path of LocalCache.
void	setProactiveCaching(boolean proactiveCaching)
void	setRecLengthCacheSize(int recLengthCacheSize)
void	setSecret(String secret) Setter for configuration based secret
void	setTouchAsync(boolean touchAsync)
void	setUploadRetries(int uploadRetries)
void	updateModifiedDateOnAccess(long before) From now on, update the modified date of an object even when accessing it.

Methods inherited from class org.apache.jackrabbit.core.data.AbstractDataStore

encodeHexString, getRecordFromReference, getReferenceFromIdentifier

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

inUse

protected Map<DataIdentifier,WeakReference<DataIdentifier>> inUse

All data identifiers that are currently in use are in this set until they are garbage collected.

uploadRetryMap

protected final Map<DataIdentifier,Integer> uploadRetryMap

In memory map to hold failed asynchronous upload DataIdentifier and its retry count. Once if all retries are exhausted or file is successfully uploaded, then corresponding entry is flushed from the map. As all failed uploads are synchronously uploaded at startup, this map is not required to be persisted.

asyncTouchCache

protected final Map<DataIdentifier,Long> asyncTouchCache

In memory map to hold in-progress asynchronous touch. Once touch is successful corresponding entry is flushed from the map.

asyncDownloadCache

protected final Map<DataIdentifier,Long> asyncDownloadCache

In memory map to hold in-progress asynchronous downloads. Once download is finished corresponding entry is flushed from the map.

recLenCache

protected Map<DataIdentifier,Long> recLenCache

In memory cache to hold DataRecord.getLength() against DataIdentifier

backend

protected Backend backend

Constructor Detail

CachingDataStore

public CachingDataStore()

Method Detail

createBackend

protected abstract Backend createBackend()

getMarkerFile

protected abstract String getMarkerFile()

init

public void init(String homeDir)
          throws javax.jcr.RepositoryException

Initialized the data store. If the path is not set, <repository home>/repository/datastore is used. This directory is automatically created if it does not yet exist. During first initialization, it upload all files from local datastore to backed and local datastore act as a local cache.

Specified by:

init in interface DataStore

Parameters:

homeDir - the home directory of the repository

Throws:

javax.jcr.RepositoryException

addRecord

public DataRecord addRecord(InputStream input)
                     throws DataStoreException

Creates a new data record in Backend. The stream is first consumed and the contents are saved in a temporary file and the SHA-1 message digest of the stream is calculated. If a record with the same SHA-1 digest (and length) is found then it is returned. Otherwise new record is created in Backend and the temporary file is moved in place to LocalCache.

Specified by:

addRecord in interface DataStore

Parameters:

input - binary stream

Returns:

CachingDataRecord

Throws:

DataStoreException - if the record could not be created.

getRecord

public DataRecord getRecord(DataIdentifier identifier)
                     throws DataStoreException

Description copied from interface: DataStore

Returns the identified data record. The given identifier should be the identifier of a previously saved data record. Since records are never removed, there should never be cases where the identified record is not found. Abnormal cases like that are treated as errors and handled by throwing an exception.

Specified by:

getRecord in interface DataStore

Overrides:

getRecord in class AbstractDataStore

Parameters:

identifier - data identifier

Returns:

identified data record

Throws:

DataStoreException - if the data store could not be accessed, or if the given identifier is invalid

getRecordIfStored

public DataRecord getRecordIfStored(DataIdentifier identifier)
                             throws DataStoreException

Get a data record for the given identifier or null it data record doesn't exist in Backend

Specified by:

getRecordIfStored in interface DataStore

Parameters:

identifier - identifier of record.

Returns:

the CachingDataRecord or null.

Throws:

DataStoreException - if the data store could not be accessed

updateModifiedDateOnAccess

public void updateModifiedDateOnAccess(long before)

Description copied from interface: DataStore

From now on, update the modified date of an object even when accessing it. Usually, the modified date is only updated when creating a new object, or when a new link is added to an existing object. When this setting is enabled, even getLength() will update the modified date.

Specified by:

updateModifiedDateOnAccess in interface DataStore

Parameters:

before - - update the modified date to the current time if it is older than this value

getAllIdentifiers

public Iterator<DataIdentifier> getAllIdentifiers()
                                           throws DataStoreException

Retrieves all identifiers from Backend.

Specified by:

getAllIdentifiers in interface DataStore

Returns:

an iterator over all DataIdentifier objects

Throws:

DataStoreException - if the list could not be read

deleteRecord

public void deleteRecord(DataIdentifier identifier)
                  throws DataStoreException

This method deletes record from Backend and then from LocalCache

Specified by:

deleteRecord in interface MultiDataStoreAware

Parameters:

identifier - data identifier

Throws:

DataStoreException - if the data store could not be accessed, or if the given identifier is invalid

deleteAllOlderThan

public int deleteAllOlderThan(long min)
                       throws DataStoreException

Description copied from interface: DataStore

Delete objects that have a modified date older than the specified date.

Specified by:

deleteAllOlderThan in interface DataStore

Parameters:

min - the minimum time

Returns:

the number of data records deleted

Throws:

DataStoreException

getLastModified

public long getLastModified(DataIdentifier identifier)
                     throws DataStoreException

Return lastModified of record from Backend assuming Backend as a single source of truth.

Throws:

DataStoreException

getLength

public long getLength(DataIdentifier identifier)
               throws DataStoreException

Return the length of record from LocalCache if available, otherwise retrieve it from Backend.

Throws:

DataStoreException

getOrCreateReferenceKey

protected byte[] getOrCreateReferenceKey()
                                  throws DataStoreException

Description copied from class: AbstractDataStore

Returns the reference key of this data store. If one does not already exist, it is automatically created in an implementation-specific way. The default implementation simply creates a temporary random key that's valid only until the data store gets restarted. Subclasses can override and/or decorate this method to support a more persistent reference key.

This method is called only once during the lifetime of a data store instance and the return value is cached in memory, so it's no problem if the implementation is slow.

Overrides:

getOrCreateReferenceKey in class AbstractDataStore

Returns:

reference key

Throws:

DataStoreException - if the key is not available

getPendingUploads

public Set<String> getPendingUploads()

deleteFromCache

public void deleteFromCache(DataIdentifier identifier)
                     throws DataStoreException

Throws:

DataStoreException

onSuccess

public void onSuccess(AsyncUploadResult result)

Description copied from interface: AsyncUploadCallback

Callback method for successful asynchronous upload.

Specified by:

onSuccess in interface AsyncUploadCallback

onFailure

public void onFailure(AsyncUploadResult result)

Description copied from interface: AsyncUploadCallback

Callback method for failed asynchronous upload.

Specified by:

onFailure in interface AsyncUploadCallback

onAbort

public void onAbort(AsyncUploadResult result)

Description copied from interface: AsyncUploadCallback

Callback method for aborted asynchronous upload.

Specified by:

onAbort in interface AsyncUploadCallback

onSuccess

public void onSuccess(AsyncTouchResult result)

Description copied from interface: AsyncTouchCallback

Callback method for successful asynchronous touch.

Specified by:

onSuccess in interface AsyncTouchCallback

onFailure

public void onFailure(AsyncTouchResult result)

Description copied from interface: AsyncTouchCallback

Callback method for failed asynchronous touch.

Specified by:

onFailure in interface AsyncTouchCallback

onAbort

public void onAbort(AsyncTouchResult result)

Description copied from interface: AsyncTouchCallback

Callback method for aborted asynchronous touch.

Specified by:

onAbort in interface AsyncTouchCallback

confirmDelete

public boolean confirmDelete(DataIdentifier identifier)

Method to confirm that identifier can be deleted from Backend

Parameters:

identifier -

Returns:

clearInUse

public void clearInUse()

Description copied from interface: DataStore

Clear the in-use list. This is only used for testing to make the the garbage collection think that objects are no longer in use.

Specified by:

clearInUse in interface DataStore

isInUse

public boolean isInUse(DataIdentifier identifier)

close

public void close()
           throws DataStoreException

Description copied from interface: DataStore

Close the data store

Specified by:

close in interface DataStore

Throws:

DataStoreException - if a problem occurred

setSecret

public void setSecret(String secret)

Setter for configuration based secret

Parameters:

secret - the secret used to sign reference binaries

setMinRecordLength

public void setMinRecordLength(int minRecordLength)

Set the minimum object length.

Parameters:

minRecordLength - the length

getMinRecordLength

public int getMinRecordLength()

Return mininum object length.

Specified by:

getMinRecordLength in interface DataStore

Returns:

the minimum size in bytes

getConfig

public String getConfig()

Return path of configuration properties.

Returns:

path of configuration properties.

setConfig

public void setConfig(String config)

Set the configuration properties path.

Parameters:

config - path of configuration properties.

getCacheSize

public long getCacheSize()

Returns:

size of LocalCache.

setCacheSize

public void setCacheSize(long cacheSize)

Set size of LocalCache.

Parameters:

cacheSize - size of LocalCache.

getPath

public String getPath()

Returns:

path of LocalCache.

setPath

public void setPath(String path)

Set path of LocalCache.

Parameters:

path - of LocalCache.

getCachePurgeTrigFactor

public double getCachePurgeTrigFactor()

Returns:

Purge trigger factor of LocalCache.

setCachePurgeTrigFactor

public void setCachePurgeTrigFactor(double cachePurgeTrigFactor)

Set purge trigger factor of LocalCache.

Parameters:

cachePurgeTrigFactor - purge trigger factor.

getCachePurgeResizeFactor

public double getCachePurgeResizeFactor()

Returns:

Purge resize factor of LocalCache.

setCachePurgeResizeFactor

public void setCachePurgeResizeFactor(double cachePurgeResizeFactor)

Set purge resize factor of LocalCache.

Parameters:

cachePurgeResizeFactor - purge resize factor.

getConcurrentUploadsThreads

public int getConcurrentUploadsThreads()

setConcurrentUploadsThreads

public void setConcurrentUploadsThreads(int concurrentUploadsThreads)

getAsyncUploadLimit

public int getAsyncUploadLimit()

setAsyncUploadLimit

public void setAsyncUploadLimit(int asyncUploadLimit)

isContinueOnAsyncUploadFailure

public boolean isContinueOnAsyncUploadFailure()

setContinueOnAsyncUploadFailure

public void setContinueOnAsyncUploadFailure(boolean continueOnAsyncUploadFailure)

getUploadRetries

public int getUploadRetries()

setUploadRetries

public void setUploadRetries(int uploadRetries)

setTouchAsync

public void setTouchAsync(boolean touchAsync)

setProactiveCaching

public void setProactiveCaching(boolean proactiveCaching)

setRecLengthCacheSize

public void setRecLengthCacheSize(int recLengthCacheSize)

getBackend

public Backend getBackend()