Class LMDirichletSimilarity


  • public class LMDirichletSimilarity
    extends LMSimilarity
    Bayesian smoothing using Dirichlet priors. From Chengxiang Zhai and John Lafferty. 2001. A study of smoothing methods for language models applied to Ad Hoc information retrieval. In Proceedings of the 24th annual international ACM SIGIR conference on Research and development in information retrieval (SIGIR '01). ACM, New York, NY, USA, 334-342.

    The formula as defined the paper assigns a negative score to documents that contain the term, but with fewer occurrences than predicted by the collection language model. The Lucene implementation returns 0 for such documents.

    • Constructor Detail

      • LMDirichletSimilarity

        public LMDirichletSimilarity​(LMSimilarity.CollectionModel collectionModel,
                                     float mu)
        Instantiates the similarity with the provided μ parameter.
      • LMDirichletSimilarity

        public LMDirichletSimilarity​(float mu)
        Instantiates the similarity with the provided μ parameter.
      • LMDirichletSimilarity

        public LMDirichletSimilarity​(LMSimilarity.CollectionModel collectionModel)
        Instantiates the similarity with the default μ value of 2000.
      • LMDirichletSimilarity

        public LMDirichletSimilarity()
        Instantiates the similarity with the default μ value of 2000.
    • Method Detail

      • score

        protected float score​(BasicStats stats,
                              float freq,
                              float docLen)
        Description copied from class: SimilarityBase
        Scores the document doc.

        Subclasses must apply their scoring formula in this class.

        Specified by:
        score in class SimilarityBase
        Parameters:
        stats - the corpus level statistics.
        freq - the term frequency.
        docLen - the document length.
        Returns:
        the score.
      • explain

        protected void explain​(Explanation expl,
                               BasicStats stats,
                               int doc,
                               float freq,
                               float docLen)
        Description copied from class: SimilarityBase
        Subclasses should implement this method to explain the score. expl already contains the score, the name of the class and the doc id, as well as the term frequency and its explanation; subclasses can add additional clauses to explain details of their scoring formulae.

        The default implementation does nothing.

        Overrides:
        explain in class LMSimilarity
        Parameters:
        expl - the explanation to extend with details.
        stats - the corpus level statistics.
        doc - the document id.
        freq - the term frequency.
        docLen - the document length.
      • getMu

        public float getMu()
        Returns the μ parameter.