Class DbDataStore

  • All Implemented Interfaces:
    DataStore, MultiDataStoreAware, DatabaseAware
    Direct Known Subclasses:

    public class DbDataStore
    extends AbstractDataStore
    implements DatabaseAware, MultiDataStoreAware
    A data store implementation that stores the records in a database using JDBC. Configuration:
     <DataStore class="">
         <param name="url" value="jdbc:postgresql:test"/>
         <param name="user" value="sa"/>
         <param name="password" value="sa"/>
         <param name="databaseType" value="postgresql"/>
         <param name="driver" value="org.postgresql.Driver"/>
         <param name="minRecordLength" value="1024"/>
         <param name="maxConnections" value="2"/>
         <param name="copyWhenReading" value="true"/>
         <param name="tablePrefix" value=""/>
         <param name="schemaObjectPrefix" value=""/>
         <param name="schemaCheckEnabled" value="true"/>

    Only URL, user name and password usually need to be set. The remaining settings are generated using the database URL sub-protocol from the database type resource file.

    JNDI can be used to get the connection. In this case, use the javax.naming.InitialContext as the driver, and the JNDI name as the URL. If the user and password are configured in the JNDI resource, they should not be configured here. Example JNDI settings:

     <param name="driver" value="javax.naming.InitialContext" />
     <param name="url" value="java:comp/env/jdbc/Test" />

    For Microsoft SQL Server 2005, there is a problem reading large BLOBs. You will need to use the JDBC driver version 1.2 or newer, and append ;responseBuffering=adaptive to the database URL. Don't append ;selectMethod=cursor, otherwise it can still run out of memory. Example database URL: jdbc:sqlserver://localhost:4220;DatabaseName=test;responseBuffering=adaptive

    By default, the data is copied to a temp file when reading, to avoid problems when reading multiple blobs at the same time.

    The tablePrefix can be used to specify a schema and / or catalog name: <param name="tablePrefix" value="ds.">

    • Field Detail


        public static final int DEFAULT_MIN_RECORD_LENGTH
        The default value for the minimum object size.
        See Also:
        Constant Field Values

        public static final String STORE_TEMP_FILE
        Write to a temporary file to get the length (slow, but always works). This is the default setting.
        See Also:
        Constant Field Values

        public static final String STORE_SIZE_MINUS_ONE
        Call PreparedStatement.setBinaryStream(..., -1)
        See Also:
        Constant Field Values

        public static final String STORE_SIZE_MAX
        Call PreparedStatement.setBinaryStream(..., Integer.MAX_VALUE)
        See Also:
        Constant Field Values
      • DIGEST

        protected static final String DIGEST
        The digest algorithm used to uniquely identify records.
      • minModifiedDate

        protected long minModifiedDate
        The minimum modified date. If a file is accessed (read or write) with a modified date older than this value, the modified date is updated to the current time.
      • url

        protected String url
        The database URL used.
      • driver

        protected String driver
        The database driver.
      • user

        protected String user
        The user name.
      • password

        protected String password
        The password
      • databaseType

        protected String databaseType
        The database type used.
      • minRecordLength

        protected int minRecordLength
        The minimum size of an object that should be stored in this data store.
      • tablePrefix

        protected String tablePrefix
        The prefix for the datastore table, empty by default.
      • schemaObjectPrefix

        protected String schemaObjectPrefix
        The prefix of the table names. By default it is empty.
      • dataSourceName

        protected String dataSourceName
        The logical name of the DataSource to use.
      • tableSQL

        protected String tableSQL
        This is the property 'table' in the [databaseType].properties file, initialized with the default value.
      • createTableSQL

        protected String createTableSQL
        This is the property 'createTable' in the [databaseType].properties file, initialized with the default value.
      • insertTempSQL

        protected String insertTempSQL
        This is the property 'insertTemp' in the [databaseType].properties file, initialized with the default value.
      • updateDataSQL

        protected String updateDataSQL
        This is the property 'updateData' in the [databaseType].properties file, initialized with the default value.
      • updateLastModifiedSQL

        protected String updateLastModifiedSQL
        This is the property 'updateLastModified' in the [databaseType].properties file, initialized with the default value.
      • updateSQL

        protected String updateSQL
        This is the property 'update' in the [databaseType].properties file, initialized with the default value.
      • deleteSQL

        protected String deleteSQL
        This is the property 'delete' in the [databaseType].properties file, initialized with the default value.
      • deleteOlderSQL

        protected String deleteOlderSQL
        This is the property 'deleteOlder' in the [databaseType].properties file, initialized with the default value.
      • selectMetaSQL

        protected String selectMetaSQL
        This is the property 'selectMeta' in the [databaseType].properties file, initialized with the default value.
      • selectAllSQL

        protected String selectAllSQL
        This is the property 'selectAll' in the [databaseType].properties file, initialized with the default value.
      • selectDataSQL

        protected String selectDataSQL
        This is the property 'selectData' in the [databaseType].properties file, initialized with the default value.
      • storeStream

        protected String storeStream
        The stream storing mechanism used.
      • copyWhenReading

        protected boolean copyWhenReading
        Copy the stream to a temp file before returning it. Enabled by default to support concurrent reads.
      • temporaryInUse

        protected List<String> temporaryInUse
        The temporary identifiers that are currently in use.
    • Constructor Detail

      • DbDataStore

        public DbDataStore()
    • Method Detail

      • addRecord

        public DataRecord addRecord​(InputStream stream)
                             throws DataStoreException
        Description copied from interface: DataStore
        Creates a new data record. The given binary stream is consumed and a binary record containing the consumed stream is created and returned. If the same stream already exists in another record, then that record is returned instead of creating a new one.

        The given stream is consumed and not closed by this method. It is the responsibility of the caller to close the stream. A typical call pattern would be:

             InputStream stream = ...;
             try {
                 record = store.addRecord(stream);
             } finally {
        Specified by:
        addRecord in interface DataStore
        stream - binary stream
        data record that contains the given stream
        DataStoreException - if the data store could not be accessed
      • deleteAllOlderThan

        public int deleteAllOlderThan​(long min)
                               throws DataStoreException
        Description copied from interface: DataStore
        Delete objects that have a modified date older than the specified date.
        Specified by:
        deleteAllOlderThan in interface DataStore
        min - the minimum time
        the number of data records deleted
      • getMinRecordLength

        public int getMinRecordLength()
        Description copied from interface: DataStore
        Get the minimum size of an object that should be stored in this data store. Depending on the overhead and configuration, each store may return a different value.
        Specified by:
        getMinRecordLength in interface DataStore
        the minimum size in bytes
      • setMinRecordLength

        public void setMinRecordLength​(int minRecordLength)
        Set the minimum object length. The maximum value is around 32000.
        minRecordLength - the length
      • getRecordIfStored

        public DataRecord getRecordIfStored​(DataIdentifier identifier)
                                     throws DataStoreException
        Description copied from interface: DataStore
        Check if a record for the given identifier exists, and return it if yes. If no record exists, this method returns null.
        Specified by:
        getRecordIfStored in interface DataStore
        identifier - data identifier
        the record if found, and null if not
        DataStoreException - if the data store could not be accessed
      • getProperty

        protected String getProperty​(Properties prop,
                                     String key,
                                     String defaultValue)
        Get the expanded property value. The following placeholders are supported: ${table}: the table name (the default is DATASTORE) and ${tablePrefix}: tablePrefix plus schemaObjectPrefix as set in the configuration
        prop - the properties object
        key - the key
        defaultValue - the default value
        the property value (placeholders are replaced)
      • convert

        protected DataStoreException convert​(String cause,
                                             Exception e)
        Convert an exception to a data store exception.
        cause - the message
        e - the root cause
        the data store exception
      • updateModifiedDateOnAccess

        public void updateModifiedDateOnAccess​(long before)
        Description copied from interface: DataStore
        From now on, update the modified date of an object even when accessing it. Usually, the modified date is only updated when creating a new object, or when a new link is added to an existing object. When this setting is enabled, even getLength() will update the modified date.
        Specified by:
        updateModifiedDateOnAccess in interface DataStore
        before - - update the modified date to the current time if it is older than this value
      • getDatabaseType

        public String getDatabaseType()
        Get the database type (if set).
        the database type
      • setDatabaseType

        public void setDatabaseType​(String databaseType)
        Set the database type. By default the sub-protocol of the JDBC database URL is used if it is not set. It must match the resource file [databaseType].properties. Example: mysql.
        databaseType -
      • getDriver

        public String getDriver()
        Get the database driver
        the driver
      • setDriver

        public void setDriver​(String driver)
        Set the database driver class name. If not set, the default driver class name for the database type is used, as set in the [databaseType].properties resource; key 'driver'.
        driver -
      • getPassword

        public String getPassword()
        Get the password.
        the password
      • setPassword

        public void setPassword​(String password)
        Set the password.
        password -
      • getUrl

        public String getUrl()
        Get the database URL.
        the URL
      • setUrl

        public void setUrl​(String url)
        Set the database URL. Example: jdbc:postgresql:test
        url -
      • getUser

        public String getUser()
        Get the user name.
        the user name
      • setUser

        public void setUser​(String user)
        Set the user name.
        user -
      • isSchemaCheckEnabled

        public final boolean isSchemaCheckEnabled()
        whether the schema check is enabled
      • setSchemaCheckEnabled

        public final void setSchemaCheckEnabled​(boolean enabled)
        enabled - set whether the schema check is enabled
      • usesIdentifier

        protected void usesIdentifier​(DataIdentifier identifier)
      • clearInUse

        public void clearInUse()
        Description copied from interface: DataStore
        Clear the in-use list. This is only used for testing to make the the garbage collection think that objects are no longer in use.
        Specified by:
        clearInUse in interface DataStore
      • getMaxConnections

        public int getMaxConnections()
        Get the maximum number of concurrent connections.
        the maximum number of connections.
      • setMaxConnections

        public void setMaxConnections​(int maxConnections)
        Set the maximum number of concurrent connections in the pool. At least 3 connections are required if the garbage collection process is used.
        maxConnections - the new value
      • getCopyWhenReading

        public boolean getCopyWhenReading()
        Is a stream copied to a temporary file before returning?
        the setting
      • setCopyWhenReading

        public void setCopyWhenReading​(boolean copyWhenReading)
        The the copy setting. If enabled, a stream is always copied to a temporary file when reading a stream.
        copyWhenReading - the new setting
      • getTablePrefix

        public String getTablePrefix()
        Get the table prefix.
        the table prefix.
      • setTablePrefix

        public void setTablePrefix​(String tablePrefix)
        Set the new table prefix. The default is empty. The table name is constructed like this: ${tablePrefix}${schemaObjectPrefix}${tableName}
        tablePrefix - the new value
      • getSchemaObjectPrefix

        public String getSchemaObjectPrefix()
        Get the schema prefix.
        the schema object prefix
      • setSchemaObjectPrefix

        public void setSchemaObjectPrefix​(String schemaObjectPrefix)
        Set the schema object prefix. The default is empty. The table name is constructed like this: ${tablePrefix}${schemaObjectPrefix}${tableName}
        schemaObjectPrefix - the new prefix
      • getDataSourceName

        public String getDataSourceName()
      • setDataSourceName

        public void setDataSourceName​(String dataSourceName)