Package org.apache.lucene.util
Class BytesRefHash
java.lang.Object
org.apache.lucene.util.BytesRefHash
BytesRefHash
is a special purpose hash-map like data-structure
optimized for BytesRef
instances. BytesRefHash maintains mappings of
byte arrays to ids (Map<BytesRef,int>) storing the hashed bytes
efficiently in continuous storage. The mapping to the id is
encapsulated inside BytesRefHash
and is guaranteed to be increased
for each added BytesRef
.
Note: The maximum capacity BytesRef
instance passed to
add(BytesRef)
must not be longer than ByteBlockPool.BYTE_BLOCK_SIZE
-2.
The internal storage is limited to 2GB total byte storage.
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic class
Manages allocation of the per-term addresses.static class
A simpleBytesRefHash.BytesStartArray
that tracks memory allocation using a privateCounter
instance.static class
-
Field Summary
Fields -
Constructor Summary
ConstructorsConstructorDescriptionBytesRefHash
(ByteBlockPool pool) Creates a newBytesRefHash
BytesRefHash
(ByteBlockPool pool, int capacity, BytesRefHash.BytesStartArray bytesStartArray) Creates a newBytesRefHash
-
Method Summary
Modifier and TypeMethodDescriptionint
Adds a newBytesRef
int
Adds a newBytesRef
with a pre-calculated hash code.int
addByPoolOffset
(int offset) Adds a "arbitrary" int offset instead of a BytesRef term.int
byteStart
(int bytesID) Returns the bytesStart offset into the internally usedByteBlockPool
for the given bytesIDvoid
clear()
void
clear
(boolean resetPool) void
close()
Closes the BytesRefHash and releases all internally used memoryint
Returns the id of the givenBytesRef
.int
Returns the id of the givenBytesRef
with a pre-calculated hash code.Populates and returns aBytesRef
with the bytes for the given bytesID.void
reinit()
reinitializes theBytesRefHash
after a previousclear()
call.int
size()
Returns the number ofBytesRef
values in thisBytesRefHash
.int[]
sort
(Comparator<BytesRef> comp) Returns the values array sorted by the referenced byte values.
-
Field Details
-
DEFAULT_CAPACITY
public static final int DEFAULT_CAPACITY- See Also:
-
-
Constructor Details
-
BytesRefHash
public BytesRefHash() -
BytesRefHash
Creates a newBytesRefHash
-
BytesRefHash
Creates a newBytesRefHash
-
-
Method Details
-
size
public int size()Returns the number ofBytesRef
values in thisBytesRefHash
.- Returns:
- the number of
BytesRef
values in thisBytesRefHash
.
-
get
Populates and returns aBytesRef
with the bytes for the given bytesID.Note: the given bytesID must be a positive integer less than the current size (
size()
)- Parameters:
bytesID
- the idref
- theBytesRef
to populate- Returns:
- the given BytesRef instance populated with the bytes for the given bytesID
-
sort
Returns the values array sorted by the referenced byte values.Note: This is a destructive operation.
clear()
must be called in order to reuse thisBytesRefHash
instance.- Parameters:
comp
- theComparator
used for sorting
-
clear
public void clear(boolean resetPool) -
clear
public void clear() -
close
public void close()Closes the BytesRefHash and releases all internally used memory -
add
Adds a newBytesRef
- Parameters:
bytes
- the bytes to hash- Returns:
- the id the given bytes are hashed if there was no mapping for the
given bytes, otherwise
(-(id)-1)
. This guarantees that the return value will always be >= 0 if the given bytes haven't been hashed before. - Throws:
BytesRefHash.MaxBytesLengthExceededException
- if the given bytes are > 2 +ByteBlockPool.BYTE_BLOCK_SIZE
-
add
Adds a newBytesRef
with a pre-calculated hash code.- Parameters:
bytes
- the bytes to hashcode
- the bytes hash codeHashcode is defined as:
int hash = 0; for (int i = offset; i < offset + length; i++) { hash = 31 * hash + bytes[i]; }
- Returns:
- the id the given bytes are hashed if there was no mapping for the
given bytes, otherwise
(-(id)-1)
. This guarantees that the return value will always be >= 0 if the given bytes haven't been hashed before. - Throws:
BytesRefHash.MaxBytesLengthExceededException
- if the given bytes are >ByteBlockPool.BYTE_BLOCK_SIZE
- 2
-
find
Returns the id of the givenBytesRef
.- See Also:
-
find
Returns the id of the givenBytesRef
with a pre-calculated hash code.- Parameters:
bytes
- the bytes to look forcode
- the bytes hash code- Returns:
- the id of the given bytes, or
-1
if there is no mapping for the given bytes.
-
addByPoolOffset
public int addByPoolOffset(int offset) Adds a "arbitrary" int offset instead of a BytesRef term. This is used in the indexer to hold the hash for term vectors, because they do not redundantly store the byte[] term directly and instead reference the byte[] term already stored by the postings BytesRefHash. See add(int textStart) in TermsHashPerField. -
reinit
public void reinit()reinitializes theBytesRefHash
after a previousclear()
call. Ifclear()
has not been called previously this method has no effect. -
byteStart
public int byteStart(int bytesID) Returns the bytesStart offset into the internally usedByteBlockPool
for the given bytesID- Parameters:
bytesID
- the id to look up- Returns:
- the bytesStart offset into the internally used
ByteBlockPool
for the given id
-