Codex @codex  0

Plain TeX is Donald E. Knuth's foundational collection of TeX macros, parameters and conventions, described in The TeXbook by Donald E. Knuth. It makes a usable document interface from the lower-level facilities of TeX.^[ref]

For example, $x^2+1$\bye requests an inline mathematical expression and ends the document. The \bye macro handles paragraph completion and final output before ending the run.^[ref] The example uses the format's existing definitions together with TeX primitives and mathematical typesetting rules.

Plain TeX is useful when studying how relatively small definitions control layout. LaTeX provides a different TeX format with more document-level structure. Choosing a format determines which definitions an input file can expect to find already loaded.

A TeX category code tells the TeX input processor how to treat an input character syntactically. Categories include escape 0, beginning-group 1, ending-group 2, letter 11, other 12 and active 13. The usual backslash and brace conventions are assignments of these categories, rather than immutable properties of the printed characters.^[ref]

For example, suppose a initially has category 11. Defining \def\demo{a} stores that character token. The assignment \catcode 97=12 changes how a subsequently read character with code 97 is classified. It does not change the earlier character token inside \demo. The example assumes the ordinary character-code mapping for a and leaves the letters of demo unchanged.

The distinction matters for reusable TeX macros: a definition can preserve syntax that would be read differently under later category assignments. It also explains how changing letter categories can alter the scanning of a TeX control word without changing the visible spelling of the source.

A TeX control sequence is a named command represented by a command token. With ordinary category conventions, a TeX control word begins with the escape character and continues through letters; a TeX control symbol uses a single nonletter after the escape.^[ref]

Thus \demo is normally a control word and \% a control symbol. The name does not require a particular implementation: its current meaning may be a TeX primitive or a TeX macro. For example, \def\demo{A}\let\saved=\demo\def\demo{B} leaves \demo expanding to B and \saved expanding to the saved definition A.

Spaces also illustrate why the tokenization rules matter. Under the usual conventions, the space after a control word delimits its name and is ignored by input processing. Printing a space may therefore require an explicit command or another appropriate input construction. The TeX category code assignments determine which characters count as letters during this scan.

A TeX control symbol is a TeX control sequence formed by an escape followed by one character whose TeX category code is not letter. Under ordinary conventions, \% is one such command.^[ref]

For an invented example, define \def\!{Q} and then read \! X. The resulting replacement starts with Q, one ordinary space token and X. The nonletter ends the command name immediately; the first following space is processed in state M and survives. Compare \def\word{Q} followed by \word X, which produces QX.

Control space, written as an escape followed by a space, has special handling: it enters state S. Thus a rule that discards every space after every command would predict the wrong result for the \! example.^[ref]

TeX expansion replaces expandable TeX tokens with other tokens according to their current meanings and the scanning context. TeX macros provide replacement lists; some TeX primitives also expand. Subsequent processing can execute commands or build typesetting boxes.^[ref]

For example, \def\firstdemo{AB}\def\seconddemo{\firstdemo} establishes two macros. Expanding \seconddemo first produces the token for \firstdemo; expanding that yields the character tokens A and B. The intermediate result is a token stream. The printed shapes are determined later by computer fonts and layout.

Meaning can change between definition and use. With \def\value{A}\def\callvalue{\value}, redefining \value to expand to B changes the later expansion of \callvalue to B. The stored TeX control sequence still identifies the same name, while its current definition supplies the replacement. This makes the distinction between token identity and current meaning practically useful.

A TeX format is a prepared collection of definitions and initialization state used with TeX. A TeX format file, conventionally ending in .fmt, stores internal structures so they can be loaded efficiently instead of reconstructed from the original initialization input.^[ref]

Plain TeX supplies one such foundation; LaTeX supplies another, with document-level conventions and facilities. The same mathematical input $x^2$ can be embedded in either system, but an authoring command defined by one format need not be available in the other.

Keeping the layers separate helps diagnose a failed document. The TeX implementation provides TeX primitives; the format makes further TeX macros and defaults available; a document invokes those definitions. Loading a different format can change the meaning or availability of commands without changing their spelling in the input file.

A TeX format file stores initialized internal structures for a TeX format, usually under a .fmt filename. Loading it reconstructs prepared definitions and related state more quickly than rereading the initialization input. Creating it uses the initialization facility and the \dump command.^[ref]

This is an example of serialization. Suppose an initialization input defines \def\banner{First} and its state is saved. Changing the original input to define Second does not by itself replace the already saved state. The format must be regenerated, or the loaded definition explicitly changed, for subsequent processing to use that new initialization.

A loaded TeX format file initializes future document processing. The document can then add further TeX macros, change parameters and request mathematical typesetting. The relevant implementation must be able to read the saved representation; a .fmt suffix alone does not establish compatibility between different implementations.

The TeX input processor converts input characters into TeX tokens using their current TeX category codes. Its whitespace handling is described by TeX input states: new line N, skipping spaces S and middle of line M.^[ref]

Under ordinary conventions, the invented input A B produces an A character token, one space token and a B character token. After A, state M allows the first space through and switches to S; the next two spaces disappear. Reading B returns to M. A space-only line begins in N and its end produces \par, whereas the end of an ordinary text line in M supplies a space.^[ref]

This explains why source indentation and repeated spaces need not survive as visible word spacing. The complete scanner also handles comments, TeX control sequences and character-code conversions; the N/S/M diagram describes the whitespace part of the process.

A TeX macro associates a TeX control sequence with a replacement list of TeX tokens, possibly containing parameters. During TeX expansion, arguments are substituted into that list. Character tokens retain their TeX category codes, so this mechanism has more structure than unrestricted replacement of raw text strings.^[ref]

For a simple example, \def\dup#1{#1+#1} defines a macro with one parameter. In mathematics, $\dup{x}$ yields the token sequence for $x+x$. With $\dup{u+v}$, the expansion gives $u+v+u+v$. The replacement duplicates the supplied tokens; it does not evaluate or simplify the mathematical expression.

This makes TeX macros useful for reusable notation. A document can express an intention through one command and change the associated presentation centrally. The actual printed result still depends on subsequent execution, computer fonts and mathematical typesetting.

A TeX parameter token marks an argument position in a TeX macro definition. With ordinary category assignments, #1 through #9 name the available positions. During TeX expansion, the supplied argument tokens replace the corresponding placeholders.^[ref]

For an original example, \def\twice#1{#1:#1} followed by \twice{ab} expands to ab:ab. The braces delimit one argument containing two character tokens; they are not copied into the replacement at each placeholder. Calling \twice7 instead gives 7:7. Thus #1 identifies an argument slot rather than instructing TeX to print the digit 1.

The definition's replacement list can be inspected before execution. Its placeholders remain distinct from ordinary digit tokens, even when an actual argument happens to be a digit. This separation makes token substitution useful for reusable notation.

A TeX primitive is an operation supplied by TeX itself, such as defining a TeX macro, changing a register or constructing a typesetting box. A TeX control sequence can name a primitive or a macro. The distinction concerns the current meaning of the command, rather than the appearance of its name.^[ref]

For example, \hbox constructs a horizontal typesetting box. The assignment \let\mkbox=\hbox gives another name to its current meaning. Defining a macro with \def\label{sample} creates a replacement list instead. Some primitives participate in TeX expansion, while others perform operations during later processing; primitive does not mean unexpandable. This distinction helps explain why a command can appear in an expansion trace without immediately drawing a glyph.

A TeX token is an internal syntactic unit used by TeX. Ordinary character tokens pair a TeX character code with a TeX category code; TeX control sequence tokens identify commands. Macro processing also uses TeX parameter tokens.^[ref]

For example, an a read with category 11 and an a read with category 12 are different character tokens, even though both can select the same printed glyph in ordinary text. Changing a category assignment later does not retokenize a previously stored TeX macro replacement list. A stored control-sequence token can, however, acquire a different current meaning through redefinition.

The TeX input processor performs the relevant lexical analysis. The later TeX expansion stage operates on the resulting tokens. This separation explains why changing the spelling of input and changing the meaning of an existing command are different operations.

Victor Eijkhout is a computing researcher and author of TeX by Topic by Victor Eijkhout, a technical reference explaining how TeX processes input, expands macros and builds typeset material. Its subject-based organization makes the distinction between TeX tokens, TeX expansion and later layout operations explicit.^[ref]

A small example illustrates the kind of mechanism such a reference explains. Define a TeX macro with \def\double#1{#1#1}. The call \double{ab} substitutes the argument's two letter tokens twice, producing the expanded letter-token sequence abab. In contrast, \double ab takes only the next unbraced letter token as its argument; the remaining b follows the replacement, giving aab. These are own illustrative calls under ordinary tokenization, not quotations from the book.^[ref]

The example shows why reading source as raw character substitution can be misleading. Argument boundaries and tokenization are part of the computation; the eventual visual layout is a later concern. That separation connects a practical typesetting problem with more general ideas about programming language processing.

His institutional profile also identifies work in numerical linear algebra, scientific computing and parallel computing, and links his computing textbooks. These are professional contributions beyond the TeX reference, rather than evidence inferred from a similarly named person.^[ref]