Package org.apache.jackrabbit.oak.spi.blob

Class AbstractBlobStore

  • All Implemented Interfaces:
    java.lang.AutoCloseable, Cache.Backend<AbstractBlobStore.BlockId,​AbstractBlobStore.Data>, BlobStore, GarbageCollectableBlobStore
    Direct Known Subclasses:
    CachingBlobStore, FileBlobStore, MemoryBlobStore
    public abstract class AbstractBlobStore
extends java.lang.Object
implements GarbageCollectableBlobStore, Cache.Backend<AbstractBlobStore.BlockId,​AbstractBlobStore.Data>
    An abstract data store that splits the binaries in relatively small blocks, so that each block fits in memory.

    Each data store id is a list of zero or more entries. Each entry is either

    • data (a number of bytes), or
    • the hash code of the content of a number of bytes, or
    • the hash code of the content of a data store id (indirect hash)
    Thanks to the indirection, blocks can be kept relatively small, so that caching is simpler, and so that the storage backend doesn't need to support arbitrary size blobs (some storage backends buffer blobs in memory) and fast seeks (some storage backends re-read the whole blob when seeking).

    The format of a 'data' entry is: type (one byte; 0 for data), length (variable size int), data (bytes).

    The format of a 'hash of content' entry is: type (one byte; 1 for hash), level (variable size int, 0 meaning not nested), size (variable size long), hash code length (variable size int), hash code.

    The format of a 'hash of data store id' entry is: type (one byte; 1 for hash), level (variable size int, nesting level), total size (variable size long), size of data store id (variable size long), hash code length (variable size int), hash code.

    • Field Detail

      • BLOCK_SIZE_LIMIT

        protected static final int BLOCK_SIZE_LIMIT
        The minimum block size. Blocks must be larger than that so that the content hash is always shorter than the data itself.
        See Also:
        Constant Field Values

      • inUse

        protected java.util.Map<java.lang.String,​java.lang.ref.WeakReference<java.lang.String>> inUse
        The blob ids that are still floating around in memory. The blob store assumes such binaries must not be deleted, because those binaries are not referenced yet in a way the garbage collection algorithm can detect (not referenced at all, or only referenced in memory).

    • Constructor Detail

      • AbstractBlobStore

        public AbstractBlobStore()

    • Method Detail

      • setBlockSizeMin

        public void setBlockSizeMin​(int x)

      • setBlockSize

        public void setBlockSize​(int x)
        Description copied from interface: GarbageCollectableBlobStore
        Set the block size used by this blob store, if the blob store splits binaries into blocks. If not, this setting is ignored.
        Specified by:
        setBlockSize in interface GarbageCollectableBlobStore
        Parameters:
        x - the block size in bytes.

      • getBlockSize

        public int getBlockSize()

      • writeBlob

        public java.lang.String writeBlob​(java.lang.String tempFilePath)
                           throws java.io.IOException
        Description copied from interface: GarbageCollectableBlobStore
        Write a blob from a temporary file. The temporary file is removed afterwards. A file based blob stores might simply rename the file, so that no additional writes are necessary.
        Specified by:
        writeBlob in interface GarbageCollectableBlobStore
        Parameters:
        tempFilePath - the temporary file name
        Returns:
        the blob id
        Throws:
        java.io.IOException

      • writeBlob

        public java.lang.String writeBlob​(java.io.InputStream in)
                           throws java.io.IOException
        Description copied from interface: BlobStore
        Write a blob from an input stream. This method closes the input stream.
        Specified by:
        writeBlob in interface BlobStore
        Parameters:
        in - the input stream
        Returns:
        the blob id
        Throws:
        java.io.IOException

      • writeBlob

        public java.lang.String writeBlob​(java.io.InputStream in,
                                  BlobOptions options)
                           throws java.io.IOException
        Default implementation ignores options and delegates to the writeBlob(InputStream) method.
        Specified by:
        writeBlob in interface BlobStore
        Parameters:
        in - the input stream to write
        options - the options to use
        Returns:
        Throws:
        java.io.IOException

      • getInputStream

        public java.io.InputStream getInputStream​(java.lang.String blobId)
                                   throws java.io.IOException
        Description copied from interface: BlobStore
        Returns a new stream for given blobId. The streams returned from multiple calls to this method are byte wise equals. That is, subsequent calls to read return the same sequence of bytes as long as neither call throws an exception.
        Specified by:
        getInputStream in interface BlobStore
        Parameters:
        blobId - the blob id
        Returns:
        a new stream for given blobId
        Throws:
        java.io.IOException

      • getReference

        public java.lang.String getReference​(@NotNull
                                     @NotNull java.lang.String blobId)
        Description copied from interface: BlobStore
        Returns a secure reference to blob referred by blobid, or null if no such reference is available.
        Specified by:
        getReference in interface BlobStore
        Parameters:
        blobId - blobId referring the blob for which reference is required
        Returns:
        binary reference, or null

      • getBlobId

        public java.lang.String getBlobId​(@NotNull
                                  @NotNull java.lang.String reference)
        Description copied from interface: BlobStore
        Returns the blobId that referred by the given binary reference. Returns null if the reference is invalid, for example if it points to a blob that does not exist.
        Specified by:
        getBlobId in interface BlobStore
        Parameters:
        reference - binary reference
        Returns:
        matching blobId, or null

      • getOrCreateReferenceKey

        protected byte[] getOrCreateReferenceKey()
        Returns the reference key of this blob store. If one does not already exist, it is automatically created in an implementation-specific way. The default implementation simply creates a temporary random key that's valid only until the data store gets restarted. Subclasses can override and/or decorate this method to support a more persistent reference key.

        This method is called only once during the lifetime of a data store instance and the return value is cached in memory, so it's no problem if the implementation is slow.

        Returns:
        reference key

      • setReferenceKey

        public void setReferenceKey​(byte[] referenceKey)

      • setReferenceKeyEncoded

        public void setReferenceKeyEncoded​(java.lang.String encodedKey)
        Set the referenceKey from Base64 encoded byte array
        Parameters:
        encodedKey - base64 encoded key

      • usesBlobId

        protected void usesBlobId​(java.lang.String blobId)

      • storeBlock

        protected abstract void storeBlock​(byte[] digest,
                                   int level,
                                   byte[] data)
                            throws java.io.IOException
        Store a block of data.
        Parameters:
        digest - the content hash (32 bytes)
        level - the indirection level (0 is for user data, 1 is a list of digests that point to user data, 2 is a list of digests that point to digests, and so on). This parameter is for informational use only, and it is not required to store it unless that's easy to achieve
        data - the data to be stored (the number of bytes is at most the block size)
        Throws:
        java.io.IOException

      • isMarkEnabled

        protected abstract boolean isMarkEnabled()

      • mark

        protected abstract void mark​(AbstractBlobStore.BlockId id)
                      throws java.lang.Exception
        Throws:
        java.lang.Exception

      • markInUse

        protected void markInUse()
                  throws java.io.IOException
        Throws:
        java.io.IOException

      • readBlob

        public int readBlob​(java.lang.String blobId,
                    long pos,
                    byte[] buff,
                    int off,
                    int length)
             throws java.io.IOException
        Description copied from interface: BlobStore
        Read a number of bytes from a blob.
        Specified by:
        readBlob in interface BlobStore
        Parameters:
        blobId - the blob id
        pos - the position within the blob
        buff - the target byte array
        off - the offset within the target array
        length - the number of bytes to read
        Returns:
        the number of bytes read
        Throws:
        java.io.IOException

      • readBlockFromBackend

        protected abstract byte[] readBlockFromBackend​(AbstractBlobStore.BlockId id)
                                        throws java.lang.Exception
        Load the block from the storage backend. Returns null if the block was not found.
        Parameters:
        id - the block id
        Returns:
        the block data, or null
        Throws:
        java.lang.Exception

      • getBlobLength

        public long getBlobLength​(java.lang.String blobId)
                   throws java.io.IOException
        Description copied from interface: BlobStore
        Get the length of the blob.
        Specified by:
        getBlobLength in interface BlobStore
        Parameters:
        blobId - the blob id
        Returns:
        the length
        Throws:
        java.io.IOException

      • mark

        protected void mark​(java.lang.String blobId)
             throws java.io.IOException
        Throws:
        java.io.IOException

      • resolveChunks

        public java.util.Iterator<java.lang.String> resolveChunks​(java.lang.String blobId)
                                                   throws java.io.IOException
        Description copied from interface: GarbageCollectableBlobStore
        Resolve chunks stored in the blob store from the given Id. This will not return any chunks stored in-line in the id.
        Specified by:
        resolveChunks in interface GarbageCollectableBlobStore
        Parameters:
        blobId - the blob id
        Returns:
        the iterator
        Throws:
        java.io.IOException - Signals that an I/O exception has occurred.

      • deleteChunks

        public boolean deleteChunks​(java.util.List<java.lang.String> chunkIds,
                            long maxLastModifiedTime)
                     throws java.lang.Exception
        Description copied from interface: GarbageCollectableBlobStore
        Deletes the blobs with the given ids.
        Specified by:
        deleteChunks in interface GarbageCollectableBlobStore
        Parameters:
        chunkIds - the chunk ids
        maxLastModifiedTime - the max last modified time to consider for retrieval, with the special value '0' meaning no filtering by time
        Returns:
        true, if successful
        Throws:
        java.lang.Exception - the exception

      • close

        public void close()
           throws java.lang.Exception
        Specified by:
        close in interface java.lang.AutoCloseable
        Throws:
        java.lang.Exception