Length-prefixed encoding

A length-prefixed encoding represents a variable-size field by writing a length before its payload. The length's units, width, endianness, permitted range and inclusion or exclusion of the prefix itself are part of the serialization format. For example, the Transport Layer Security presentation language counts vector lengths in bytes and encodes variable-length vectors with a length field whose width follows the specified upper bound.^[ref]

An original two-field format uses one unsigned byte for each payload length:Here $|x|$ counts payload bytes and $\Vert$ is concatenation. The ASCII fields ab and c give 02 61 62 01 63, while a and bc give 01 61 02 62 63. Bare concatenation would give 61 62 63 in both cases. This distinction is useful when a cryptographic hash function or message authentication code must commit to two separate fields.

The encoding is an injective function on its stated input set. Read the first length byte $n$, then exactly $n$ bytes for $x$; read the next length byte $k$, then exactly $k$ bytes for $y$. This reconstructs both fields uniquely. For a complete two-field record, require that no bytes remain. Missing bytes, absent length fields or forbidden lengths are rejected rather than silently reinterpreted. The proof depends on these parsing rules, not merely on putting some number before the data.

A fixed-width length field also imposes a representable limit: 256 payload bytes cannot be encoded by the one-byte scheme above. Truncating that length modulo 256 would make the parser see zero and destroy the intended format. A larger format must specify a larger prefix or a different encoding.

Finally, byte count and text-character count can differ. The text é has two bytes c3 a9 in UTF-8, so its one-byte length prefix here is 02. A length prefix provides boundaries; cryptographic domain separation may additionally require a field type, role label or protocol version to distinguish records with different meanings.

A fixed-width length field uses a fixed number of encoded bytes to state a payload length. The payload itself can vary in size. For an unsigned $k$-byte field in big-endian endianness, bytes $b_0,\dots ,b_{k-1}$ representThe format must additionally define the length unit, permitted range and whether headers count toward it. TLS uses unsigned, network-order integers and gives variable-length vectors a prefix wide enough for their declared maximum; its vector length counts bytes, not elements.^[ref]

For an original two-byte length-prefixed encoding, prefix 02 01 means $2\cdot 256+1=513$ payload bytes in big-endian order. The complete field then occupies $515$ bytes. The same two prefix bytes would mean $258$ under little-endian interpretation, changing the boundary of every following field. The byte order is therefore part of the serialization, not a display preference.

A two-byte field can represent at most $65535$; a format permitting payload lengths only up to $400$ must reject larger encoded values even though they fit the field. Conversely, a $300$-byte payload needs prefix 01 2c under this convention. Truncating it to one byte would declare only $44$ bytes. A format change cannot be implemented by silently discarding high-order bits.

Length-prefix validation before allocation checks the declared range and available input before reserving space or consuming the payload. A complete decoder also specifies how truncation and trailing bytes are handled. Byte count and character count differ for encodings such as UTF-8. A length prefix supplies boundaries; a field's role or type may still need binding through cryptographic domain separation.

Length-prefix validation before allocation checks that an encoded payload length is permitted and safely usable before requesting memory allocation based on it. A fixed-width length field bounds what the encoding can represent; an application can impose a smaller limit. TLS vectors, for example, have explicitly specified minimum and maximum byte lengths.^[ref]

For an original decoder model, let $n$ be the decoded unsigned payload length, $L$ the application limit and $R$ the bytes remaining in a complete input buffer after the prefix. Before allocating or consuming that payload, requireCheck that the complete prefix is available before reading it. For a two-byte big-endian prefix 02 01, $n=513$. With $L=400$ and $R=300$, the input fails both checks. Prefix 01 2c instead declares $300$ bytes and satisfies these two bounds. The example counts only payload bytes in $n$.

Arithmetic needs its own bound. If an allocation includes $h$ additional bytes and the size type's maximum value is $M$, verify $h\le M$ and $n\le M-h$ before forming $n+h$. Computing the sum first can already cause integer overflow. Similarly, when allocating $n$ elements of positive size $s$, check $n\le \lfloor M/s\rfloor$ before multiplication.

For a streaming decoder, $n>R$ can mean that more input is needed rather than that the final message is malformed. It still does not justify consuming unavailable bytes or allocating beyond the stated resource limit. Nested fields need bounds relative to their enclosing field. These checks establish safe structural handling of a length-prefixed encoding; they do not authenticate the sender or validate the payload's meaning.