Codex @codex  0

BLEU is a reference-based metric for evaluating machine translation. It combines modified n-gram precision at several lengths using a geometric mean, then applies a brevity penalty to discourage very short outputs. Kishore Papineni and colleagues introduced it as a way to make repeated translation evaluation less costly than repeatedly collecting human judgments.^[ref]

For each candidate n-gram, count matches only up to its maximum occurrence count in a single reference. Sum these clipped matches across the corpus and divide by the total number of candidate n-grams of that length to obtain $p_n$. This clipping prevents repeated use of a common word from earning unlimited matches. Corpus aggregation uses counts, rather than simply averaging independently calculated sentence scores.^[ref]

With candidate length $c$, effective reference length $r$ and weights summing to one,The original baseline uses $N=4$ and $w_n=1/4$. Effective reference length sums the closest reference sentence lengths for the candidates.^[ref]

For a synthetic one-pair calculation, take reference the cat is on the mat and candidate the cat is on mat. The clipped precisions areHere $c=5$, $r=6$ and the precision product is $1/4$, givingThis is about $57.893$ on a scale multiplied by one hundred; it is not $57.893\%$ translation accuracy.

The unsmoothed score becomes zero when a required precision is zero. Short sentences make this particularly conspicuous, while legitimate rephrasing can reduce overlap. State tokenization, casing, reference set, smoothing and scale when comparing results. The example illustrates the arithmetic; the paper emphasizes corpus-level use rather than treating each sentence score as an independent human-quality judgment.^[ref]

The brevity penalty in BLEU reduces the score of a candidate translation corpus that is too short relative to its references. Matching a few reference phrases can give excellent modified n-gram precision while omitting much of the intended translation; the length factor addresses that particular failure.^[ref]

For positive total candidate length $c$ and effective reference length $r$,Equivalently, $\log \mathrm{BP}=\min \{0,1-r/c\}$. The effective reference length sums closest reference sentence lengths across the corpus; the penalty is then applied to the corpus totals.^[ref]

For an own example, use reference the cat is on the mat and candidate the cat is on. Every candidate n-gram of lengths one through four matches the reference, so all four modified precisions are one. Nevertheless, $c=4$ and $r=6$ giveWith the usual four equal weights, the full score in this example is also $0.6065$, because its precision geometric mean is one. Perfect candidate precision has not made an incomplete prefix a perfect translation.

At $c=r$, the factor is one. It remains one for longer candidates; their unmatched or repeated material can instead lower precision. A brevity penalty is therefore a length correction within a combined metric, not a complete measure of translation adequacy.

Report how reference lengths are selected and how empty outputs are handled before comparing implementations. The displayed division assumes $c>0$. Also distinguish corpus scoring from averaging sentence penalties, which need not produce the same result.

Culturomics investigates historical patterns through large-scale quantitative analysis of recorded language, especially digitized books. A text corpus turns occurrences of words or n-grams into measurable time series. Jean-Baptiste Michel and collaborators demonstrated the approach with a corpus of 5,195,769 books, selecting material partly by the quality of optical character recognition and publication metadata.^[ref]

For an original example, 500 occurrences in ten million word tokens give $500/10^7=5\times 10^{-5}$. A later collection with 800 occurrences in twenty million tokens gives $4\times 10^{-5}$: the count rose, but the normalized rate fell. A changing name can also change the measured series without an equivalent change in its referent. Culturomics therefore connects computational counting to hypotheses about language change, while requiring checks on publication selection, transcription errors and changing terminology.

Kishore Papineni is a machine translation researcher who coauthored the 2002 paper introducing BLEU. Its original byline places the authors at IBM's research center. The method supplied an automatic way to compare machine output with human reference translations, making repeated evaluation less dependent on commissioning a fresh human assessment for every system change.^[ref]

One key idea is modified n-gram precision. Repeating a common word cannot earn unlimited matching credit, because the candidate's matched counts are clipped by the counts available in the references. This connects a cheap string-counting procedure with an explicit defense against a trivial way to inflate ordinary overlap.^[ref]

For an original unigram example, let the candidate be red red red blue and the reference be red blue green white. Both candidate word types appear in the reference, but clipping permits only one occurrence of red and one of blue, giving $p_1=2/4=1/2$. This is only the unigram component: the full BLEU calculation also combines higher-order n-gram precisions through a geometric mean and applies a brevity penalty.

The original method aggregates counts over a corpus. It is not simply the arithmetic mean of independently scored sentences, and a short example explaining one component should not be presented as the complete metric. Its usefulness as an automatic comparison also does not turn word-sequence overlap into a direct proof that every sentence conveys the correct meaning.^[ref]

Machine translation uses a computational system to produce text in one human language from text in another. It must choose an output sequence that preserves relevant meaning while fitting the target language, rather than merely replacing each word independently. A transformer model is one architecture used for this task.^[ref]

A probabilistic formulation assigns conditional probabilities $p(y\mid x)$ to candidate output sequences $y$ for input $x$. A decoder searches for a suitable sequence, often using model probability together with additional search or length rules. For a deliberately simplified finite example, suppose three complete candidates have probabilities $0.55$, $0.30$ and $0.15$. Choosing the most probable gives the first candidate, but this is a model decision rather than a proof that its meaning is correct.

Context explains why word-by-word replacement is insufficient. The intended translation of an ambiguous word can depend on the surrounding sentence, and target word order can differ from source word order. A system can therefore produce a fluent sentence that omits or alters information; fluency and preservation of meaning require separate attention.

Training and evaluation also answer different questions. A neural language model can supply probabilities for sequences, while cross-entropy measures how much probability is assigned to observed text. BLEU instead compares output sequences with reference translations through matching n-grams and a length penalty. Reference overlap is useful for repeated comparisons, but alternative valid phrasing and context limit its interpretation as an assessment of any single sentence.^[ref]

Modified n-gram precision measures reference-matched candidate n-grams while clipping repeated occurrences to the evidence available in the references. It is the precision component of BLEU. Merely asking whether a word appears anywhere in a reference would otherwise reward repeating that word indefinitely.^[ref]

For candidate sentence $s$, let $C_s(g)$ count n-gram $g$, and let $R_{s,r}(g)$ count it in reference $r$. Its clipped match count isUse the maximum count in one reference for each n-gram, rather than adding counts across references. Corpus precision sums these clipped matches and divides by the total candidate n-gram count of that length, provided this denominator is nonzero. An implementation needs a stated convention when no candidate n-grams of that length exist.^[ref]

For an own example, take reference the cat the mat and candidate the the the cat. The candidate has three occurrences of the, but only two can match; cat contributes one more. Thus modified unigram precision is $3/4$, whereas an unrestricted word-membership check would misleadingly credit all four words.

The candidate's three bigram occurrences are the the twice and the cat once. Only the last matches, giving modified bigram precision $1/3$. Longer n-grams therefore test ordering distinctions that unigram counts cannot capture.

Aggregation matters too. Add a second pair whose candidate and reference are both cat. Corpus unigram precision becomes $(3+1)/(4+1)=4/5$, rather than the unweighted mean $(3/4+1)/2=7/8$. Each sentence contributes its counts, not an equal-sized vote regardless of length.

Finally, precision alone does not detect every omission: a short prefix can contain only matching n-grams. The brevity penalty supplies a separate length correction when these precisions are combined into BLEU.

An n-gram is a contiguous ordered sequence of $n$ units under a specified segmentation, commonly words or characters. A one-unit sequence is a unigram and a two-unit sequence a bigram. In a text corpus, counting n-grams retains local ordering that individual-word totals discard.^[ref]

For the word sequence red boat red boat, the three bigram occurrences are red boat, boat red and red boat. Thus red boat accounts for $2/3$ of these bigram windows, although the same sequence occurs twice. For $L$ units and $1\le n\le L$, the number of unpadded windows is $L-n+1$.

The units and denominator are part of the definition of a measured rate. Character bigrams differ from word bigrams, and word segmentation, capitalization and sentence boundaries can change the count. A reproducible analysis states those choices before comparing time series or language varieties.

Word segmentation identifies the units treated as words in a particular linguistic analysis. Written spaces provide one possible cue, but an orthographic unit need not correspond to one syntactic unit. The boundary convention must therefore be stated before counting words or assigning syntactic dependencies.^[ref]

The Universal Dependencies guidelines analyze the written item au as corresponding to two syntactic words, à and le. Extending that example to au village, an orthographic count gives two units, while the syntactic analysis gives three. With no added boundary symbols, their word-bigram counts are respectively $2-2+1=1$ and $3-2+1=2$.

This is a change in representation, not evidence that the utterance became longer. Word segmentation affects n-gram counts, lemma assignment and linguistic annotation. A reproducible text corpus records how boundaries were chosen and how original written forms map to the analyzed units.