Kishore Papineni
= Kishore Papineni
{c}
<Kishore Papineni> is a <machine translation> researcher who coauthored the 2002 paper introducing <BLEU>. Its original byline places the authors at IBM's research center. The method supplied an automatic way to compare machine output with human reference translations, making repeated evaluation less dependent on commissioning a fresh human assessment for every system change.<BLEU: a Method for Automatic Evaluation of Machine Translation by Kishore Papineni and others>[Page311: original byline, affiliation, Abstract and motivation]{ref}
One key idea is <modified n-gram precision>. Repeating a common word cannot earn unlimited matching credit, because the candidate's matched counts are clipped by the counts available in the references. This connects a cheap string-counting procedure with an explicit defense against a trivial way to inflate ordinary overlap.<BLEU: a Method for Automatic Evaluation of Machine Translation by Kishore Papineni and others>[Section2.1: clipped n-gram counts]{ref}
For an original unigram example, let the candidate be `red red red blue` and the reference be `red blue green white`. Both candidate word types appear in the reference, but clipping permits only one occurrence of `red` and one of `blue`, giving $p_1=2/4=1/2$. This is only the unigram component: the full <BLEU> calculation also combines higher-order <n-gram> precisions through a <geometric mean> and applies a <brevity penalty>.
The original method aggregates counts over a corpus. It is not simply the <arithmetic mean> of independently scored sentences, and a short example explaining one component should not be presented as the complete metric. Its usefulness as an automatic comparison also does not turn word-sequence overlap into a direct proof that every sentence conveys the correct meaning.<BLEU: a Method for Automatic Evaluation of Machine Translation by Kishore Papineni and others>[Section2.1.1: corpus aggregation; Sections2.2.2–2.3: brevity penalty and combined score]{ref}