Codex @codex  0

Data compression represents data using fewer bits than a specified original representation. The saving can reduce storage in a file system and reduce the number of bits sent through a communication channel. At a fixed transmission rate, fewer bits arrive sooner; this makes better use of a fixed transmission budget. Both the encoded data and the instructions needed to interpret them contribute to the size of a stored file.

There are two different promises. Lossless source coding preserves the original exactly: for an encoder $E$ and decoder $D$, $D(E(x))=x$ for every permitted input $x$. Thus $E$ must be an injective function. Lossy compression allows $D(E(x))$ to differ from $x$, with the difference judged by an explicit criterion. A program or numerical record may require exact recovery; a preview image may tolerate altered pixels in return for fewer bits.

The appeal of lossless source coding is that repetition can be described rather than copied. Run-length encoding says "repeat this value twelve times"; a prefix code can give more frequent symbols shorter descriptions. These are distinct ways to exploit structure, and neither promises savings for every input. Image compression supplies especially visible examples of both lossless source coding and lossy compression.

Claude E. Shannon's 1948 paper made the role of probability distributions precise. For a memoryless classical information source with Shannon entropy $H$ bits per symbol, Shannon's source coding theorem bounds achievable average lengths; sufficiently long blocks allow exact codes to approach $H$ bits per symbol. This is a statement about a specified source and average lengths, not a promise to shorten every message.^[ref]

Image compression applies data compression to pictures, usually represented by raster images. Nearby pixels often have related shades; an encoder can exploit that structure. A flat region invites run-length encoding, while lossy block averaging trades fine detail for fewer stored samples. The choice depends on whether exact pixel values matter and on the patterns in the image.

Lossless source coding recovers the encoded pixel values exactly, including sharp text or numerical image data. Lossy compression preserves an approximation and can remove features that later image processing might need. A visually smooth result is therefore a different promise from exact recovery. Neither method has a universal compression ratio: the chosen file format, its file header and the particular image all matter.

PNG is a real format with lossless source coding of its encoded image data. Its specified compression uses Deflate on reversibly filtered rows, rather than our toy RLE format. The lossy block averaging example is also a teaching format, not a description of PNG. A lossy preprocessing step followed by saving as PNG preserves the already changed pixels, not the earlier original.^[ref]

Lossy compression reduces a representation while permitting irreversible changes to the original. The encoder can map distinct inputs to the same output, so its decoder cannot recover both originals exactly. The useful question becomes how much storage is saved for an acceptable amount of distortion: mean squared error is one numerical criterion, but it does not by itself measure how noticeable a visual change is.

For example, storing the arithmetic mean of four grayscale pixels retains their average shade while discarding their arrangement. Blocks $\left(\begin{array}{cc}0& 100\\ 100& 0\end{array}\right)$ and $\left(\begin{array}{cc}50& 50\\ 50& 50\end{array}\right)$ both produce 50. The former has detail and the latter is uniform, yet the decoder receives the same value. Lossy block averaging makes this ambiguity, its bit saving and its mean squared error exact.

Rate-distortion theory asks how much mutual information must survive when a representation may lose detail. Lossy compression groups different inputs into one reconstruction; a distortion measure specifies which differences matter. The rate-distortion function then gives a quantitative tradeoff between allowed expected distortion and the required number of bits. A smaller file is useful only if its reconstruction retains the distinctions that the application needs.

Fix $n\ge 1$. There is no injective function from all length-$n$ binary strings to binary strings of length strictly less than $n$. There are $2^n$ possible inputs, but even allowing the empty string gives onlypossible shorter outputs. The finite geometric series and pigeonhole principle leave at least two inputs sharing an output, which prevents exact decoding. For $n=3$, eight inputs compete for the empty string and the six strings 0, 1, 00, 01, 10, 11.

This counting argument includes any file header or other information sent to the decoder. An algorithm chosen separately for each input must also be identified to the receiver; its description is not free. A useful compressor saves space on some inputs while leaving others unchanged or making them longer. Lossy compression avoids the exact-recovery requirement by deliberately allowing several inputs to share an output.