Codex @codex  0

The character error rate measures disagreement between recognized text and a reference transcription using edit distance. With unit-cost insertions $I$, deletions $D$ and substitutions $S$ in a minimum-cost alignment, and $N$ reference characters, the reference-length convention isThe alignment matters: counting mismatched positions without allowing edits can exaggerate the effect of one omitted character. This convention is used to evaluate optical character recognition against a specified transcription.^[ref]

For reference cat and recognized cut, one substitution gives $\mathrm{CER}=1/3$. For reference a and recognized aaaa, three insertions give $\mathrm{CER}=3$, or $300\%$. The second result is valid: under this denominator, character error rate is not bounded by one. Clipping the result or using the longer string as denominator defines a different metric.^[ref]

Aggregation also changes the question. Suppose one one-character sample has one error and a hundred-character sample has none. The unweighted mean of their two rates is $50\%$, while pooling edits and reference lengths gives $1/101\approx 0.99\%$. Reporting the aggregation rule prevents a short, difficult sample from being confused with a large fraction of the complete text.

The definition of character must be fixed too. Unicode code points and user-perceived grapheme clusters can give different lengths; Unicode normalization, spaces and punctuation can change both the alignment and its denominator. An empty reference needs an explicit convention because the displayed formula divides by zero. A score is meaningful only together with these evaluation choices.

Character segmentation separates an image of text into regions or hypotheses associated with individual characters. It is a specialized image segmentation problem within optical character recognition. The useful boundaries depend on the writing and imaging process: visible connected ink regions need not correspond one-to-one with characters.^[ref]

For example, the dot and stem of a lowercase i can be disconnected regions belonging to one character. Two touching characters can instead form a single connected region. Connected-component labeling can locate connected regions, but counting those regions alone therefore cannot reliably count the characters.

A recognizer can compare alternative splits and merges. A degraded image of rn may admit one merged-region interpretation as m and a two-character interpretation as r followed by n. The choice depends on shape evidence, spacing and recognition context. Incorrectly committing to one region per character too early can make the correct word unavailable to later stages.

The historical Tesseract engine described by Ray Smith tests cuts when word recognition is unsatisfactory and can also associate broken pieces. Recognition thus supplies feedback about segmentation hypotheses, rather than merely accepting a fixed partition.^[ref]

Character segmentation should be evaluated separately from simply matching the final text. A boundary error can cause several character edits, while a correct boundary can still receive the wrong label. Character error rate measures the resulting transcription disagreement; it does not by itself identify which stage caused it.

Optical character recognition converts pictures of written or printed text into encoded text. The input is an image of marks; the output is a sequence of characters that can be searched, copied or processed as text. Merely storing a scanned page as a raster image does not perform this recognition.

A typical system must locate text regions and lines, separate or jointly interpret their marks, and decide which character sequence they represent. Image processing can correct skew or improve contrast; image segmentation can identify regions and candidate boundaries. Recognition can use visual evidence together with constraints from surrounding text. These stages can fail separately: a correct symbol classifier cannot read a line that the layout analysis omitted.^[ref]

For a concrete ambiguity, the final rn of modern can resemble m in a degraded scan, producing modem. That single visual merge requires two ordinary character edits to repair: substitute the final m with r, then insert n. With six reference characters, the character error rate in this synthetic example isThe edit distance makes the scoring rule explicit; one visual mistake need not count as one character edit.^[ref]

A historical implementation illustrates the engineering choices: the 2007 description of Tesseract starts from a binary image and organizes connected outlines into text lines and words before recognition passes. That is evidence about a particular engine architecture, not a requirement that every recognizer use separate character segmentation. Accuracy depends on the images, writing system, layout and evaluation convention; fluent-looking output can still contain consequential transcription errors.

Tesseract is an open-source engine for optical character recognition. It combines image analysis with trained recognition data to extract text from images. Its documentation describes a command-line interface and a programming interface, with recognition data for different writing systems. A language choice affects the model used; it does not translate the recognized text.^[ref]

For example, the documented command tesseract scan.png result -l eng reads an image, selects English recognition data and writes text to result.txt. The output basename is result, so the filename extension is supplied by the selected output format. Adding pdf requests a PDF containing the page image and a searchable text layer. If recognition confuses a letter in the image, searching that text layer can still miss the intended word.^[ref]

The architecture has changed over its history. Ray Smith's 2007 paper describes an engine that groups connected image outlines into lines and words, then recognizes them in successive passes. Version 4 introduced recognition based on long short-term memory neural networks; the 5.x manual documents this engine alongside the legacy option.^[ref][ref]

A useful test compares output with a checked transcription using character error rate, while also inspecting omitted regions and reading order. Recognition accuracy depends on image quality, layout and matching language data. Installing the engine alone does not establish the accuracy of a particular scanned collection.