Modified n-gram precision measures reference-matched candidate n-grams while clipping repeated occurrences to the evidence available in the references. It is the precision component of BLEU. Merely asking whether a word appears anywhere in a reference would otherwise reward repeating that word indefinitely.[ref]
For candidate sentence , let count n-gram , and let count it in reference . Its clipped match count is
Use the maximum count in one reference for each n-gram, rather than adding counts across references. Corpus precision sums these clipped matches and divides by the total candidate n-gram count of that length, provided this denominator is nonzero. An implementation needs a stated convention when no candidate n-grams of that length exist.[ref]
For an own example, take reference the cat the mat and candidate the the the cat. The candidate has three occurrences of the, but only two can match; cat contributes one more. Thus modified unigram precision is , whereas an unrestricted word-membership check would misleadingly credit all four words.
The candidate's three bigram occurrences are the the twice and the cat once. Only the last matches, giving modified bigram precision . Longer n-grams therefore test ordering distinctions that unigram counts cannot capture.
Aggregation matters too. Add a second pair whose candidate and reference are both cat. Corpus unigram precision becomes , rather than the unweighted mean . Each sentence contributes its counts, not an equal-sized vote regardless of length.
Finally, precision alone does not detect every omission: a short prefix can contain only matching n-grams. The brevity penalty supplies a separate length correction when these precisions are combined into BLEU.

Articles by others on the same topic (0)

There are currently no matching articles.