= TeX token
{c}
A \b[TeX token] is an internal syntactic unit used by <TeX>. Ordinary character tokens pair a <TeX character code> with a <TeX category code>; <TeX control sequence> tokens identify commands. Macro processing also uses <TeX parameter tokens>.<TeX by Topic by Victor Eijkhout>[§2.4: character, control-sequence and internal parameter tokens]{ref}
For example, an `a` read with category 11 and an `a` read with category 12 are different character tokens, even though both can select the same printed <glyph> in ordinary text. Changing a category assignment later does not retokenize a previously stored <TeX macro> replacement list. A stored control-sequence token can, however, acquire a different current meaning through redefinition.
The <TeX input processor> performs the relevant <lexical analysis>. The later <TeX expansion> stage operates on the resulting tokens. This separation explains why changing the spelling of input and changing the meaning of an existing command are different operations.
Back to article page
