Character error rate ($\mathrm{CER}$)

The character error rate measures disagreement between recognized text and a reference transcription using edit distance. With unit-cost insertions $I$, deletions $D$ and substitutions $S$ in a minimum-cost alignment, and $N$ reference characters, the reference-length convention isThe alignment matters: counting mismatched positions without allowing edits can exaggerate the effect of one omitted character. This convention is used to evaluate optical character recognition against a specified transcription.^[ref]

For reference cat and recognized cut, one substitution gives $\mathrm{CER}=1/3$. For reference a and recognized aaaa, three insertions give $\mathrm{CER}=3$, or $300\%$. The second result is valid: under this denominator, character error rate is not bounded by one. Clipping the result or using the longer string as denominator defines a different metric.^[ref]

Aggregation also changes the question. Suppose one one-character sample has one error and a hundred-character sample has none. The unweighted mean of their two rates is $50\%$, while pooling edits and reference lengths gives $1/101\approx 0.99\%$. Reporting the aggregation rule prevents a short, difficult sample from being confused with a large fraction of the complete text.

The definition of character must be fixed too. Unicode code points and user-perceived grapheme clusters can give different lengths; Unicode normalization, spaces and punctuation can change both the alignment and its denominator. An empty reference needs an explicit convention because the displayed formula divides by zero. A score is meaningful only together with these evaluation choices.