Package org.apache.jackrabbit.oak.index.indexer.document.flatfile.linkedList

Class PersistedLinkedListV2

  • All Implemented Interfaces:
    NodeStateEntryList
    public class PersistedLinkedListV2
extends Object
implements NodeStateEntryList
    A persistent linked list that internally uses the MVStore. This list keeps an in-memory cache, writing to the persistent store the nodes only when the cache is full. The in-memory cache is limited by two parameters:
    • cacheSize: the maximum number of elements to keep in the in-memory cache
    • cacheSizeMB: the maximum size of the in-memory cache in MB

    The recommended configuration is to rely on the total memory usage to limit the cache, giving as much memory as available in the JVM, and setting a very high limit for the number of elements. A cache miss has a very high cost, so it should be avoided as much as possible.

    Each element is stored either in the cache or in the persistent store, but not in both. And elements are not moved between the two tiers, so even if there is a cache miss, that element will remain in the persistent store. For the access pattern of the indexer, this policy has a lower rate of cache misses than if we move to the cache an element after a miss.

    To understand why, let's assume we want to traverse the children of a node P that is at line/position 100 in the FFS. When we call getChildren on P, this creates an iterator that scans from position 100 for all the children. If we call recursively getChildren on a child C of P, this will also create a new iterator that will start also at position 100. Therefore, the iterators will frequently scan from 100 down. Let's assume the cache can only hold 10 nodes and that we use a policy of moving to the cache the last node that was accessed. Then, if an iterator scans from 100 to, let's say 150, when it finishes iterating, the nodes 141 to 150 will be the only ones in the cache. The next iterator that is scanning from 100 will have cache misses for all the nodes until 140. And this will repeat for every new iterator. On the other hand, if we keep in the cache the nodes from 100 to 109, every iterator starting from 100 will at least have 10 cache hits, which is better than having a cache miss for all elements.