Class TieredMergePolicy
- All Implemented Interfaces:
Closeable
,AutoCloseable
,Cloneable
LogByteSizeMergePolicy
, except this merge
policy is able to merge non-adjacent segment, and
separates how many segments are merged at once (setMaxMergeAtOnce(int)
) from how many segments are allowed
per tier (setSegmentsPerTier(double)
). This merge
policy also does not over-merge (i.e. cascade merges).
For normal merging, this policy first computes a "budget" of how many segments are allowed to be in the index. If the index is over-budget, then the policy sorts segments by decreasing size (pro-rating by percent deletes), and then finds the least-cost merge. Merge cost is measured by a combination of the "skew" of the merge (size of largest segment divided by smallest segment), total merge size and percent deletes reclaimed, so that merges with lower skew, smaller size and those reclaiming more deletes, are favored.
If a merge will produce a segment that's larger than
setMaxMergedSegmentMB(double)
, then the policy will
merge fewer segments (down to 1 at once, if that one has
deletions) to keep the segment size under budget.
NOTE: this policy freely merges non-adjacent
segments; if this is a problem, use LogMergePolicy
.
NOTE: This policy always merges by byte size
of the segments, always pro-rates by percent deletes,
and does not apply any maximum segment size during
forceMerge (unlike LogByteSizeMergePolicy
).
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionprotected static class
Holds score and explanation for a single candidate merge.Nested classes/interfaces inherited from class org.apache.lucene.index.MergePolicy
MergePolicy.DocMap, MergePolicy.MergeAbortedException, MergePolicy.MergeException, MergePolicy.MergeSpecification, MergePolicy.MergeTrigger, MergePolicy.OneMerge
-
Field Summary
FieldsFields inherited from class org.apache.lucene.index.MergePolicy
DEFAULT_MAX_CFS_SEGMENT_SIZE, maxCFSSegmentSize, noCFSRatio, writer
-
Constructor Summary
ConstructorsConstructorDescriptionSole constructor, setting all settings to their defaults. -
Method Summary
Modifier and TypeMethodDescriptionvoid
close()
Release all resources for the policy.Determine what set of merge operations is necessary in order to expunge all deletes from the index.findForcedMerges
(SegmentInfos infos, int maxSegmentCount, Map<SegmentCommitInfo, Boolean> segmentsToMerge) Determine what set of merge operations is necessary in order to merge to <= the specified segment count.findMerges
(MergePolicy.MergeTrigger mergeTrigger, SegmentInfos infos) Determine what set of merge operations are now necessary on the index.double
Returns the current floorSegmentMB.double
Returns the current forceMergeDeletesPctAllowed setting.int
Returns the current maxMergeAtOnce setting.int
Returns the current maxMergeAtOnceExplicit setting.double
Returns the current maxMergedSegmentMB setting.double
double
Returns the current segmentsPerTier setting.protected TieredMergePolicy.MergeScore
score
(List<SegmentCommitInfo> candidate, boolean hitTooLarge, long mergingBytes) Expert: scores one merge; subclasses can override.setFloorSegmentMB
(double v) Segments smaller than this are "rounded up" to this size, ie treated as equal (floor) size for merge selection.setForceMergeDeletesPctAllowed
(double v) When forceMergeDeletes is called, we only merge away a segment if its delete percentage is over this threshold.setMaxMergeAtOnce
(int v) Maximum number of segments to be merged at a time during "normal" merging.setMaxMergeAtOnceExplicit
(int v) Maximum number of segments to be merged at a time, during forceMerge or forceMergeDeletes.setMaxMergedSegmentMB
(double v) Maximum sized segment to produce during normal merging.setReclaimDeletesWeight
(double v) Controls how aggressively merges that reclaim more deletions are favored.setSegmentsPerTier
(double v) Sets the allowed number of segments per tier.toString()
Methods inherited from class org.apache.lucene.index.MergePolicy
clone, getMaxCFSSegmentSizeMB, getNoCFSRatio, isMerged, setIndexWriter, setMaxCFSSegmentSizeMB, setNoCFSRatio, size, useCompoundFile
-
Field Details
-
DEFAULT_NO_CFS_RATIO
public static final double DEFAULT_NO_CFS_RATIODefault noCFSRatio. If a merge's size is >= 10% of the index, then we disable compound file for it.
-
-
Constructor Details
-
TieredMergePolicy
public TieredMergePolicy()Sole constructor, setting all settings to their defaults.
-
-
Method Details
-
setMaxMergeAtOnce
Maximum number of segments to be merged at a time during "normal" merging. For explicit merging (eg, forceMerge or forceMergeDeletes was called), seesetMaxMergeAtOnceExplicit(int)
. Default is 10. -
getMaxMergeAtOnce
public int getMaxMergeAtOnce()Returns the current maxMergeAtOnce setting.- See Also:
-
setMaxMergeAtOnceExplicit
Maximum number of segments to be merged at a time, during forceMerge or forceMergeDeletes. Default is 30. -
getMaxMergeAtOnceExplicit
public int getMaxMergeAtOnceExplicit()Returns the current maxMergeAtOnceExplicit setting.- See Also:
-
setMaxMergedSegmentMB
Maximum sized segment to produce during normal merging. This setting is approximate: the estimate of the merged segment size is made by summing sizes of to-be-merged segments (compensating for percent deleted docs). Default is 5 GB. -
getMaxMergedSegmentMB
public double getMaxMergedSegmentMB()Returns the current maxMergedSegmentMB setting.- See Also:
-
setReclaimDeletesWeight
Controls how aggressively merges that reclaim more deletions are favored. Higher values will more aggressively target merges that reclaim deletions, but be careful not to go so high that way too much merging takes place; a value of 3.0 is probably nearly too high. A value of 0.0 means deletions don't impact merge selection. -
getReclaimDeletesWeight
public double getReclaimDeletesWeight() -
setFloorSegmentMB
Segments smaller than this are "rounded up" to this size, ie treated as equal (floor) size for merge selection. This is to prevent frequent flushing of tiny segments from allowing a long tail in the index. Default is 2 MB. -
getFloorSegmentMB
public double getFloorSegmentMB()Returns the current floorSegmentMB.- See Also:
-
setForceMergeDeletesPctAllowed
When forceMergeDeletes is called, we only merge away a segment if its delete percentage is over this threshold. Default is 10%. -
getForceMergeDeletesPctAllowed
public double getForceMergeDeletesPctAllowed()Returns the current forceMergeDeletesPctAllowed setting.- See Also:
-
setSegmentsPerTier
Sets the allowed number of segments per tier. Smaller values mean more merging but fewer segments.NOTE: this value should be >= the
setMaxMergeAtOnce(int)
otherwise you'll force too much merging to occur.Default is 10.0.
-
getSegmentsPerTier
public double getSegmentsPerTier()Returns the current segmentsPerTier setting.- See Also:
-
findMerges
public MergePolicy.MergeSpecification findMerges(MergePolicy.MergeTrigger mergeTrigger, SegmentInfos infos) throws IOException Description copied from class:MergePolicy
Determine what set of merge operations are now necessary on the index.IndexWriter
calls this whenever there is a change to the segments. This call is always synchronized on theIndexWriter
instance so only one thread at a time will call this method.- Specified by:
findMerges
in classMergePolicy
- Parameters:
mergeTrigger
- the event that triggered the mergeinfos
- the total set of segments in the index- Throws:
IOException
-
score
protected TieredMergePolicy.MergeScore score(List<SegmentCommitInfo> candidate, boolean hitTooLarge, long mergingBytes) throws IOException Expert: scores one merge; subclasses can override.- Throws:
IOException
-
findForcedMerges
public MergePolicy.MergeSpecification findForcedMerges(SegmentInfos infos, int maxSegmentCount, Map<SegmentCommitInfo, Boolean> segmentsToMerge) throws IOExceptionDescription copied from class:MergePolicy
Determine what set of merge operations is necessary in order to merge to <= the specified segment count.IndexWriter
calls this when itsIndexWriter.forceMerge(int)
method is called. This call is always synchronized on theIndexWriter
instance so only one thread at a time will call this method.- Specified by:
findForcedMerges
in classMergePolicy
- Parameters:
infos
- the total set of segments in the indexmaxSegmentCount
- requested maximum number of segments in the index (currently this is always 1)segmentsToMerge
- contains the specific SegmentInfo instances that must be merged away. This may be a subset of all SegmentInfos. If the value is True for a given SegmentInfo, that means this segment was an original segment present in the to-be-merged index; else, it was a segment produced by a cascaded merge.- Throws:
IOException
-
findForcedDeletesMerges
public MergePolicy.MergeSpecification findForcedDeletesMerges(SegmentInfos infos) throws IOException Description copied from class:MergePolicy
Determine what set of merge operations is necessary in order to expunge all deletes from the index.- Specified by:
findForcedDeletesMerges
in classMergePolicy
- Parameters:
infos
- the total set of segments in the index- Throws:
IOException
-
close
public void close()Description copied from class:MergePolicy
Release all resources for the policy.- Specified by:
close
in interfaceAutoCloseable
- Specified by:
close
in interfaceCloseable
- Specified by:
close
in classMergePolicy
-
toString
-