Codex @codex  0

A combining character is an encoded text character intended to combine with another character in its written presentation. A familiar case is U+0301 COMBINING ACUTE ACCENT: the sequence e followed by that code point represents an accented letter without requiring a separate precomposed code point for the whole letter.^[ref]

The same acute-accent code point can mark a distinct letter, stress or tone in different languages. Those functions do not require a different encoding of the accent in every language. Text shaping determines how the selected glyph is attached to the base shape, and glyph positioning can place additional marks relative to earlier marks.

A combining sequence and a grapheme cluster are related but distinct notions. The default segmentation rules determine which code points an editor treats as one unit. Unicode normalization addresses equivalence between encodings, such as a precomposed accented letter and a base-plus-accent sequence; it does not perform the geometric placement of their glyphs.

A computer font stores data used to display text: glyph descriptions, spacing measurements, and mappings from text characters to glyphs. An OpenType computer font organizes these into separate tables. Its cmap table maps encoded text characters to default glyph indices; other tables can specify subsequent substitutions and positioning.

The separation permits several representations of a glyph. An outline font stores scalable boundaries, whereas a computer font can also contain images already sampled into pixels. The rendering system uses the selected representation and placement data to produce the visible text. A file containing glyph outlines therefore supplies more than a list of Unicode numbers.^[ref]

A glyph is a visual form used to display written content. A text character is the encoded abstract item; a glyph is the shape chosen for its presentation. One text character can have several glyphs, such as contextual forms in Arabic, while several text characters can be displayed by one typographic ligature.

For example, the two text characters “f” and “i” may be drawn separately or replaced by an “fi” typographic ligature. The underlying text can remain the same in both cases. A glyph index identifies a form within a particular computer font; it is not a portable Unicode code point.^[ref]

A glyph advance is the displacement of the logical pen position after a glyph is processed. In horizontal layout its main component is often called the advance width. The OpenType horizontal-metrics table records an advanceWidth in font units, together with a left side bearing.^[ref]

The advance need not equal the width of the visible glyph's bounding box. A spacing text character can select a glyph with a positive advance and no visible outline. Conversely, a drawn shape can extend beyond its nominal advance, or a positioned accent can have a zero advance. The geometry of the ink and the progression of the logical pen therefore require separate data. In a simple horizontal metric, the advance is the left side bearing plus the ink width plus the right side bearing; overhangs can make a bearing negative.

For a hypothetical pair with advances 600 and 600 font units, a pair adjustment of minus 80 gives a total advance of 1120 font units. The second origin moves 80 font units left of its unadjusted position. No glyph outline needs to change for this kerning operation. The final layout can therefore have different advances from the default values stored for isolated glyphs.

A glyph index is a numerical identifier for a glyph inside a particular computer font. It identifies an entry in that font's own glyph collection, rather than a Unicode code point. In an OpenType computer font, the cmap table maps character codes to default glyph indices; unavailable characters map to index zero, whose special .notdef glyph indicates missing coverage.^[ref]

For a hypothetical example, one computer font might map U+0041 to index 37 while another maps the same character A to index 9. Both can display the correct letter. Carrying index 37 from the first computer font into the second can select an unrelated form. A later glyph substitution can also change the selected glyph index while leaving the underlying text characters unchanged. This distinction matters in rendering caches and when exchanging shaped text between systems.

Glyph substitution replaces selected glyph indices by other glyph indices while laying out text. In OpenType, the GSUB table specifies these replacements after the default character-to-glyph index mapping. A substitution can replace one glyph by another, several by a typographic ligature, or one by several; its application can depend on surrounding glyphs.^[ref]

For a hypothetical computer font, suppose f and i initially select indices17 and23. A matching ligature rule could replace the pair by index91. These numbers belong to that particular computer font; they do not change the stored Unicode sequence into code point91. The example also shows why substitution differs from kerning: replacing the pair selects a new shape, whereas adjusting the pair's positions can retain both original shapes.

Lookup order matters when one substitution creates or removes the input pattern for a later substitution. Text shaping must therefore interpret the applicable OpenType rules, rather than independently choosing a default glyph for each text character.

A grapheme cluster is a sequence of Unicode code points treated as a unit by a text-segmentation rule. Default extended grapheme clusters approximate user-perceived characters, supporting operations such as moving a text cursor or deleting a visible unit. They are not a universal linguistic definition of a text character; boundaries can require language-specific tailoring.^[ref]

For example, e followed by the combining character U+0301 COMBINING ACUTE ACCENT contains two code points, but forms one default extended grapheme cluster. Its UTF-8 representation has three bytes: 65 CC 81 in hexadecimal notation. The precomposed U+00E9 also forms one grapheme cluster, with two UTF-8 bytes, C3 A9. The two representations correspond under the canonical rules of Unicode normalization.^[ref]

A glyph count answers another question: text shaping can represent the two text characters f and i by one typographic ligature even though their default grapheme cluster boundary remains. Byte offsets, code point offsets, grapheme cluster boundaries and glyph indices therefore serve different purposes.

Level-based reordering of bidirectional text is the final positional reversal procedure of the Unicode Bidirectional Algorithm after directional levels and line boundaries have been resolved. RuleL2 processes thresholds from the largest level down through the smallest level whose value is an odd integer. Each pass reverses every maximal consecutive run whose levels meet or exceed the threshold.^[ref]

An original trace supplies the resolved levels rather than attempting to infer them. Take the logical itemsAt threshold2, reverse only the two middle digits, obtaining $(a,R_1,R_2,2,1,R_3,b)$. At threshold1, reverse the entire middle five-item run, givingThe digits underwent two reversals, preserving their internal order. The $R_i$ labels each denote one abstract right-to-left text character; every array is written in physical left-to-right order.

Thresholds include intermediate values that no item happens to have. For example, a pair at level4 surrounded by level1 items participates in passes4,3,2 and then the enclosing pass1. Omitting pass3 changes the result, as the bidirectional embedding level example demonstrates.

An implementation may carry levels with the items, as the figure generator does. It can also locate each reversal range from the original fixed level array: earlier passes permute items only inside higher-level runs, preserving the boundaries of every later lower-threshold run. Keeping logical indices on the displayed items supplies the eventual position map. Line-final whitespace resetting belongs to ruleL1, and combining-mark handling and mirroring require their separate rules.^[ref]

Logical text order is the order in which text characters are represented in a stored sequence for interpretation. A bidirectional renderer derives visual text order from that sequence; it does not require the stored characters to be prearranged in screen-position order.^[ref]

An original indexing example uses six abstract items. Write the stored positions in order asThe $R_i$ labels each stand for a single right-to-left text character. All displayed arrays here are read physically from left to right. Applying level-based reordering of bidirectional text to these supplied levels gives the visual list $a,R_2,1,2,R_1,b$, whose source-position list is $0,4,2,3,1,5$. Stored position1 remains position1 even though its item is drawn near the right end.

This distinction makes a text-processing task precise. An operation on logical positions1 through4 selects the contiguous stored subsequence $R_1,1,2,R_2$. A renderer may need a position map to relate that range to its displayed cells. Sorting characters by screen coordinate before searching or serializing the text would change the sequence being processed.

Logical order does not determine a font's glyph geometry. Text shaping, line layout and the Unicode Bidirectional Algorithm contribute different information. For example, a different line break can change which items are reordered together while the underlying stored sequence remains the same.^[ref]

OpenType is a table-based computer font format separating glyph representations, character mapping, measurements, naming metadata and optional layout rules. Its cmap table selects default glyph indices for encoded text characters; substitution and positioning tables support more elaborate text shaping.

An OpenType computer font can contain TrueType outlines or Compact Font Format outlines, and the specification also provides for bitmap and other glyph representations. The file format therefore does not prescribe a single kind of outline geometry. Extensions such as .ttf and .otf are useful conventions, but the actual table contents determine the available representation.^[ref]

This organization lets one system answer separate questions: which glyph represents this text, how much space should it occupy, and how should it be drawn at this size? A language's contextual forms can change the first answer without changing the original Unicode text.

A text character is an abstract unit of written information, distinguished from its visible glyph and from the bytes used to store it. A Unicode code point identifies an encoded character; UTF-8 specifies a byte representation. Neither counting bytes nor counting code points always counts what a reader perceives as individual characters.^[ref]

For example, an accented letter can be represented by a base letter followed by a combining accent. The sequence can be treated as one grapheme cluster, while a computer font and text shaping determine its displayed form. Conversely, the two text characters “f” and “i” can remain distinct in stored text while appearing as one typographic ligature. This is why a text editor's cursor movement, a byte offset and a glyph index answer different questions.

Text shaping turns a sequence of Unicode text characters into a layout of glyphs from a chosen computer font. The result includes glyph indices, positions and glyph advances. A text character therefore does not determine a visible shape independently of its neighbors, writing system and selected computer font.^[ref]

For example, the stored sequence f followed by i can become a single typographic ligature. Contextual Arabic forms illustrate a different requirement: a letter's neighbors influence the selected glyph. Glyph substitution chooses forms, while glyph positioning places the chosen forms; kerning adjusts their spacing. These operations can preserve the original Unicode text, so copying text should not be implemented by reading glyph indices as if they were code points.^[ref]

Text shaping precedes rasterization: first determine what shapes to draw and where; then determine the pixels that represent those shapes at the requested size.

A typographic ligature combines two or more written forms into a single glyph. A familiar example is an “fi” shape joining the end of “f” to the following “i”. Its use can improve spacing or avoid a collision between the two shapes.

An OpenType shaping system can first map the separate text characters “f” and “i” to their default glyph indices, then apply a glyph substitution that replaces that pair by one ligature glyph index. The stored text need not be replaced by a special ligature character. Selection and copying can therefore preserve the original two-character sequence even though the display contains one glyph.^[ref]

A typographic ligature also need not be produced merely by moving two glyphs closer together: substitution changes the selected form, whereas positioning changes where selected forms are drawn.

Typography concerns the selection and arrangement of written forms. In digital systems, the visible result depends on several distinct operations: interpreting text characters, selecting glyphs from a computer font, positioning them, and converting their geometry into pixels through rasterization.

This distinction matters when a document looks wrong. Missing glyphs, incorrect glyph substitutions and poorly rendered pixels can produce different failures even when the underlying Unicode text is unchanged.^[ref]

The Unicode Bidirectional Algorithm computes visual text order from logical text order. Unicode bidirectional classes and context determine bidirectional embedding levels: levels with an even integer value run left-to-right, and those with an odd integer value run right-to-left. Stored code points retain their order; text shaping is separate.^[ref]

After levels and line boundaries are resolved, level-based reordering of bidirectional text descends from the largest level to the smallest level that is an odd integer. Each pass reverses maximal consecutive runs at or above its level.^[ref]

Here is a small example with the resolved levels supplied. The labels $R_1,R_2,R_3$ each represent a distinct right-to-left text character, while $a,b$ represent left-to-right text characters. All rows below are written in physical left-to-right order. Start withThe level 2 pass changes the middle digits to $2,1$. The level 1 pass then reverses the whole five-item middle run, producingThe digits have been reversed twice, preserving their original internal order. Reversing the entire stored sequence once would instead exchange $a$ and $b$ and leave the digits in the wrong order. This example demonstrates the reordering step, not how all Unicode bidirectional classes are resolved.

A bidirectional isolate limits directional interaction between inserted text and its surroundings. Markup can express isolation without storing explicit formatting code points.^[ref]

Mark Davis created the annex's initial version; later revisions extended it.^[ref]