Hamming distortion ($d(x,\hat{x})=1_{\{x\ne \hat{x}\}}$)

Hamming distortion is zero for a correctly reconstructed symbol and one for a mismatch. Its expected distortion is the symbol-error probability:For strings of length $n$, separable distortion divides their Hamming distance by $n$. Thus $001$ reconstructed as $000$ has Hamming distance one and average Hamming distortion $1/3$. Counting only whether the whole string is correct would give a different criterion.

Let $X_1,X_2,X_3$ be independent fair bits. The encoder sends their majority bit: zero when at least two inputs are zero, and one otherwise. The decoder reconstructs $000$ or $111$ by repeating that received bit three times. This is lossy compression: different original strings deliberately share a reconstruction.

There are eight equally likely inputs. The strings $000$ and $111$ have zero Hamming distance from their reconstructions; each of the other six has Hamming distance one. One transmitted bit represents three source symbols, soEvery input has average Hamming distortion at most $1/3$, whereas the population average is $1/4$. The actual block code therefore obeys $R\ge R(D)\approx 0.18872$, but does not reach the optimal long-block rate.

This construction uses the same reconstruction strings as the threefold binary repetition code, but the tasks differ. An error-correcting encoder expands one source bit into three transmitted bits; this lossy compression example compresses three source bits into one transmitted bit. The source distribution and the direction of the mapping matter.

For a stationary source of independent and identically distributed fair bits, with Hamming distortion and binary reconstruction alphabet,The binary entropy convention is $0{\log }_{2}0=0$. Once $D\ge 1/2$, always reconstructing zero meets the distortion constraint without sending any bits. The formula $1-h_2(D)$ must not be extended to $D>1/2$: a looser constraint cannot increase the required rate.^[ref]

The short argument identifies both sides of the optimum. For any feasible binary reconstruction, put $E=X\oplus \hat{X}$, using exclusive or. Knowing $\hat{X}$ makes $X$ and $E$ determine one another, so the conditional entropy obeyswhen $0\le D\le 1/2$. The last step uses the increase of binary entropy on that interval. Hence $I(X;\hat{X})=1-H(X\mid \hat{X})\ge 1-h_2(D)$. Conversely, choose $\hat{X}=X\oplus N$ with an independent Bernoulli random variable $N$ of parameter $D$. This binary symmetric channel has expected distortion $D$ and mutual information $1-h_2(D)$, attaining the bound.