N-gram (source code)
= N-gram
An <n-gram> is a contiguous ordered sequence of $n$ units under a specified segmentation, commonly words or characters. A one-unit sequence is a unigram and a two-unit sequence a bigram. In a <text corpus>, counting <n-grams> retains local ordering that individual-word totals discard.<Google Books Ngram Viewer information>[What does the Ngram Viewer do?, unigram/bigram examples and normalization]{ref}
For the word sequence `red boat red boat`, the three bigram occurrences are `red boat`, `boat red` and `red boat`. Thus `red boat` accounts for $2/3$ of these bigram windows, although the same sequence occurs twice. For $L$ units and $1\le n\le L$, the number of unpadded windows is $L-n+1$.
The units and denominator are part of the definition of a measured rate. Character bigrams differ from word bigrams, and <word segmentation>, capitalization and sentence boundaries can change the count. A reproducible analysis states those choices before comparing <time series> or <language> varieties.
Back to article page
