= Modified n-gram precision
{title2=$p_n=\frac{\sum_{s,g}\min\{C_s(g),\max_rR_{s,r}(g)\}}{\sum_{s,g}C_s(g)}$}
<Modified n-gram precision> measures reference-matched candidate <n-grams> while clipping repeated occurrences to the evidence available in the references. It is the precision component of <BLEU>. Merely asking whether a word appears anywhere in a reference would otherwise reward repeating that word indefinitely.<BLEU: a Method for Automatic Evaluation of Machine Translation by Kishore Papineni and others>[§2.1 and footnote 2, clipped occurrence counts]{ref}
For candidate sentence $s$, let $C_s(g)$ count <n-gram> $g$, and let $R_{s,r}(g)$ count it in reference $r$. Its clipped match count is
$$
M_s(g)=\min\!\left\{C_s(g),\max_r R_{s,r}(g)\right\}.
$$
Use the maximum count in one reference for each <n-gram>, rather than adding counts across references. Corpus precision sums these clipped matches and divides by the total candidate <n-gram> count of that length, provided this denominator is nonzero. An implementation needs a stated convention when no candidate <n-grams> of that length exist.<BLEU: a Method for Automatic Evaluation of Machine Translation by Kishore Papineni and others>[§§2.1–2.1.1, reference maxima and corpus aggregation]{ref}
For an own example, take reference `the cat the mat` and candidate `the the the cat`. The candidate has three occurrences of `the`, but only two can match; `cat` contributes one more. Thus modified unigram precision is $3/4$, whereas an unrestricted word-membership check would misleadingly credit all four words.
The candidate's three bigram occurrences are `the the` twice and `the cat` once. Only the last matches, giving modified bigram precision $1/3$. Longer <n-grams> therefore test ordering distinctions that unigram counts cannot capture.
Aggregation matters too. Add a second pair whose candidate and reference are both `cat`. Corpus unigram precision becomes $(3+1)/(4+1)=4/5$, rather than the unweighted mean $(3/4+1)/2=7/8$. Each sentence contributes its counts, not an equal-sized vote regardless of length.
Finally, precision alone does not detect every omission: a short prefix can contain only matching <n-grams>. The <brevity penalty> supplies a separate length correction when these precisions are combined into <BLEU>.
Back to article page
