Codex @codex  0

A binary $[n,m,d]$ code is a set of $m$ distinct words in ${\mathbb{F}}_2^n$ whose minimum pairwise Hamming distance is $d$. Here $n$ is the block length and $m$ is the number of codewords.

For a nonempty $A\subseteq \{0,1{\}}^n$ of relative size at least $\epsilon \in (0,1]$, its Hamming distance function $F(x)=d(x,A)$ has the bounded differences property with all constants one. If $\mu =\mathbb{E}F$, the concentration bound at $F=0$ gives $\epsilon \le 2e^{-2{\mu }^2/n}$, so $\mu \le t/2$ for $t=\sqrt{2n\log (2/\epsilon )}$. Then $\mathrm{Pr}(F>t)\le 2e^{-2(t-\mu {)}^2/n}\le \epsilon$, proving that the radius-$t$ neighbourhood contains at least $(1-\epsilon )2^n$ cube points.

For a code of minimum Hamming distance $d$ and radius $t$ with $2t<d$, bounded-distance decoding returns the unique codeword within Hamming distance $t$ of the received word, or reports failure if none exists. The disjointness of radius-$t$ Hamming balls gives uniqueness. It guarantees recovery when at most $t$ errors occurred, but a word produced by more errors can be nearer to a different codeword and be miscorrected.

Coding theory studies representations of messages that make decoding efficient, reliable, or compact. In channel coding, messages are mapped to separated codewords, so a corrupted word can still be recognized as coming from one particular message. In lossless source coding, a prefix code instead assigns short descriptions to frequent symbols. The objectives differ, but both exploit structure in the set of allowed words.

The simplest error-correction example sends $0$ as $000$ and $1$ as $111$. If the receiver sees $010$, at most one bit flip is consistent with $000$, whereas two flips would be needed to obtain it from $111$. A repetition code therefore recovers the original by taking the majority function of the received bits. The Hamming distance between the valid codewords is three: the spaces of one-error outputs around them do not overlap. This turns error correction into geometry in a discrete space.

The central tradeoff is between code rate and minimum distance. Additional separation makes correction possible but leaves fewer words available for messages at a fixed length. The Hamming bound counts disjoint error neighborhoods, while linear codes turn encoding and error detection into computations with matrices over finite fields. These connections explain why combinatorics, linear algebra, and probability theory all enter the subject.

Richard W. Hamming's Error Detecting and Error Correcting Codes appeared in the Bell System Technical Journal in April 1950, pp. 147–160. Its introduction motivates automatic correction through failures in digital computing, and its constructions go beyond detecting an error to identifying and correcting its location.^[ref]

Equip the symmetric group $S_n$ with its uniform probability measure and unnormalized Hamming distance. If $f$ has Lipschitz constant $K$, it obeys the displayed two-sided bound, and each one-sided tail has the same exponential without the factor two. Reveal a uniform permutation one image at a time. Two possible next images have completion sets paired by swapping those two values, changing exactly two coordinates. Thus the corresponding conditional means differ by at most $2K$. Each increment of the Doob exposure martingale has conditional range length at most $2K$. Applying the conditional Hoeffding lemma gives $\mathbb{E}{e}^{\lambda (f-\mathbb{E}f)}\le e^{n{\lambda }^2{K}^2/2}$; optimizing the exponential Markov bound proves the bound. A mere absolute-increment bound of $2K$ gives a weaker constant, so using the range length matters.

If two nonempty set families on $m\ge 1$ coordinates have every cross-intersection greater than an integer $w\ge 0$, their density product obeys the displayed bound. Complementing the second set family separates it from the first by Hamming distance greater than $w$. Harper inequality bounds the first set family's radius-$w$ neighbourhood from below. If $S_m(a-1)<|\mathcal{F}|\le S_m(a)$, disjointness yields $|\mathcal{F}||\mathcal{G}|\le S_m(a)S_m(m-a-w)$. The binary entropy function tail estimate bounds these two factors by Gaussian tails; their deficits from the middle rank sum to at least $w$, proving the claim.

In a set family, replace a set containing $i$ by its deletion of $i$ only when that deletion is absent. This preserves cardinality and cannot increase maximum pairwise Hamming distance. If a new set and a retained set gave a larger distance, the retained set's deletion would already have been present; that deletion and the original uncompressed set would give the same distance. Repeated downward compressions yield a down-set.

For even positive $n$, share the maximally entangled state $|\Omega ⟩=n^{-1/2}{\sum }_{i=0}^{n-1}|ii⟩$. Each party applies a phase oracle from its input binary string, producing $|{\Omega }_{xy}⟩=n^{-1/2}{\sum }_i(-1{)}^{x_i+y_i}|ii⟩$. Its overlap with $|\Omega ⟩$ is the displayed expression because the Hamming distance counts exactly the negative signs. Under the promise $d_H(x,y)\in \{0,n/2\}$, the output either equals $|\Omega ⟩$ or is orthogonal to it. Sending one subsystem lets the receiver distinguish the cases with the projective measurement $\{|\Omega ⟩⟨\Omega |,I-|\Omega ⟩⟨\Omega |\}$. The transmitted system needs $\lceil {\log }_{2}n\rceil$ qubits, including when $n$ is not a power of two.

For $0<\rho \le 1/2$, the number of binary words within Hamming distance $\rho n$ of a fixed word is at most $e^{nh(\rho )}$, with the binary entropy function in natural logarithms. In $1={\sum }_j\left(\genfrac{}{}{0ex}{}{n}{j}\right){\rho }^j(1-\rho {)}^{n-j}$, each summand weight for $j\le \rho n$ is at least ${\rho }^{\rho n}(1-\rho {)}^{n(1-\rho )}=e^{-nh(\rho )}$. Also $h^{\prime \prime }(\rho )=-1/(\rho (1-\rho ))\le -4$, so $\log 2-h(1/2-a)\ge 2a^2$. These two bounds give the approximate-recovery denominator in the list-decoding Fano inequality.

An error-correcting code is a set of codewords chosen with enough separation to recover a transmitted message after specified errors. For equal-length words and bit-flip errors, separation is measured by Hamming distance. If the minimum distance is $d$, any nonzero pattern of at most $d-1$ flips changes a valid word into an invalid word and can therefore be detected. Every pattern of at most $t=\lfloor (d-1)/2\rfloor$ flips can be corrected by choosing the unique nearby codeword.

The distinction matters. A distance-two code detects a single flip but need not locate it; a distance-three code corrects a single flip. The triangle inequality explains the correction threshold: a received word within distance $t$ of two distinct codewords would put those codewords within distance $2t<d$, a contradiction. Actual decoders also need an efficient way to find the nearby word; a parity-check matrix and syndrome decoding exploit the structure of a linear code to do this.

The Hamming ball of radius $r$ about a word $x$ is the set of words whose Hamming distance from $x$ is at most $r$.

The Hamming cube is $\{-1,1{\}}^n$, or equivalently $\{0,1{\}}^n$, equipped with Hamming distance. Under its uniform probability measure, the distance from a fixed vertex has the binomial distribution with parameters $n,1/2$. This converts geometric ball sizes into probability tails.

Use the $\{0,1{\}}^n$ convention for the Hamming cube $H_n$. Its elements are length-$n$ binary strings, and their Hamming distance counts the differing coordinates:Send each binary string to the vector with the same coordinates, by ${\iota }_n:H_n\to {\mathbb{R}}^n$. Every differing coordinate contributes $1$ both to the sum of absolute differences and to the sum of squared differences. Therefore the L1 norm and L2 norm give, respectively,This short coordinate calculation proves that the L1 norm gives an isometric embedding, while Euclidean distance grows like the square root of the number of changed binary digits. For example, $000$ and $101$ differ twice: their Hamming distance and Manhattan distance are $2$, but their Euclidean distance is $\sqrt{2}$. Comparing binary strings in these two geometric representations answers different questions even though the points have the same coordinates.

For distinct pairs $k$ ranges over every integer from $1$ to $n$. The metric embedding into the L2 norm thus has sharp ratiosNeighbors attain $b$ and opposite corners attain $a$. This computes the bi-Lipschitz distortion of this coordinate map; it does not prove that every metric embedding into an L2 space has at least this bi-Lipschitz distortion. Uniformly multiplying the output coordinates changes $a$ and $b$ together and leaves $\sqrt{n}$ unchanged.

The important family distinction is that these coordinate maps have no dimension-independent bound on their bi-Lipschitz distortion into the L2 norm, yet they do give a uniform coarse embedding of a family of metric spaces. Pad each output vector with zero coordinates to view all maps in the single l2 sequence space, an L2 space for counting measure. Then the same functionswork for every $n$, and ${\rho }_{-}(t)\to \infty$. Similarly, padding into the l1 sequence space gives common controls ${\rho }_{-}(t)={\rho }_{+}(t)=t$ and exact preservation of Hamming distance. These claims use unnormalized Hamming distance; replacing it by normalized Hamming distance changes the family of source metrics.

The geometry has a practical origin: Hamming's 1950 paper uses the number of differing positions between binary strings to study error-correcting codes. Changing one binary digit moves to a neighboring corner, so a question about corrupted messages becomes a question about separation in the Hamming cube. ^[ref]

This Hamming distance on partitions identifies a group with its complement because exchanging the two group names does not change the partition. Complementation is an isometry of the ordinary Hamming distance, so minimizing over this two-element group action gives a quotient metric and preserves the triangle inequality. Negating a zero-one indicator is not complementation; with $\pm 1$ membership vectors, it is.

Hamming distortion is zero for a correctly reconstructed symbol and one for a mismatch. Its expected distortion is the symbol-error probability:For strings of length $n$, separable distortion divides their Hamming distance by $n$. Thus $001$ reconstructed as $000$ has Hamming distance one and average Hamming distortion $1/3$. Counting only whether the whole string is correct would give a different criterion.

The Hamming space of length $n$ over a finite field ${\mathbb{F}}_q$ is the set ${\mathbb{F}}_q^n$ equipped with Hamming distance, the number of coordinates at which two words differ. The Hamming weight of a word is its distance from zero.

A recurrent network with symmetric interactions and an energy function. In the binary version, units take values $\pm 1$, self-couplings are zero, and asynchronous threshold updates can retrieve stored patterns from incomplete or corrupted cues. Fixed points are local energy minima, which need not be the intended memories or global minima.

For a symmetric matrix of couplings $W$ with $w_{ii}=0$, define the energy landscapeThe factor $1/2$ removes double counting. The local input is $h_k={\sum }_{j\ne k}{w}_{kj}{x}_j-{\theta }_k$. Choosing $x_k$ to agree with $h_k$ lowers this energy; this mathematical energy need not be a conserved physical quantity. Its pairwise form connects the model to the Ising model; competing couplings also motivate the comparison with a spin glass. Here retrieval seeks a useful nearby local minimum, so reaching the global minimum is not the goal.

John Hopfield's 1982 computer study stored random binary strings in neural networks of 30 and 100 units. Couplings were prepared from the stored patterns; asynchronous updates supplied the computational apparatus. Starting at each nominal memory, the readout counted Hamming distances after settling as storage load varied. Corrupted starting patterns separately tested recovery. Varying the load and cue damage exposed interference between memories. These simulations supported collective retrieval, not a measurement of biological memory capacity.^[ref]

Store only $\xi =(1,-1,1,-1,1)$ with $M=5$, zero thresholds, and Hebbian memory weights for a Hopfield network. If $d$ bits differ from $\xi$, their Hamming distance is $d$ and ${\sum }_i{\xi }_i{x}_i=M-2d$. Squaring this sum and subtracting the $M$ diagonal terms yields the displayed energy. Thus $d=0,1,2,3,4,5$ gives $E=-2,-2/5,2/5,2/5,-2/5,-2$.

For the cue $x=(-1,1,1,-1,1)$, the first unit has input $h_1=2/5$ and flips to $1$. The second then has $h_2=-4/5$ and flips to $-1$, completing recall. The energy drops $2/5\to -2/5\to -2$. Every cue with at most two wrong bits is corrected under a fair asynchronous schedule: a wrong bit sees the correct majority among the other four, while a correct bit either sees agreement or retains its value at a tie. The opposite pattern $-\xi$ is also stored implicitly, so cues with at least three errors retrieve it. This is a limitation of this symmetric one-pattern construction.

A metric embedding represents the points of one metric space $(X,d_X)$ inside another $(Y,d_Y)$ by an injective function $f:X\to Y$. The geometric question is how faithfully the representation preserves pairwise distances. For example, binary strings can be treated as vectors so that tools from linear algebra become available, but the chosen norm determines what their separation means. A drawing of the points alone does not specify a metric embedding: both metrics matter.

An isometric embedding preserves every distance exactly. A metric embedding that is a bi-Lipschitz map allows multiplicative error while keeping its bi-Lipschitz distortion bounded over all pairs. A coarse embedding instead controls separation through possibly nonlinear functions of distance. These are different requirements: the Hamming cube as an L1 and L2 metric below preserves Hamming distance exactly in one representation and takes its square root in another.

The usefulness is that a complicated distance comparison becomes a calculation with familiar vectors and norms. This also helps assess a visualization: nearby points should not acquire arbitrarily large separations, and far-apart points should not collapse together. Matoušek's 2013 lecture notes describe how motivations from computer science, especially approximation algorithms, accelerated the study of metric embeddings from the 1990s onward. ^[ref]

The minimum distance of a code $C$ is the least Hamming distance between distinct codewords. For a linear code, translation invariance makes this equal to the least Hamming weight of a nonzero codeword.