Codex @codex  0

Partition $(0,1)$ into $n$ bins of width $\delta =1/n$, and let the discrete random variable $Q_n$ record the bin containing $X$. This is a quantization of $X$. Its probability mass function isand its discrete information entropy is $H_e(Q_n)=-{\sum }_j{p}_j\ln p_j$. Let ${\bar{f}}_j=p_j/\delta$ be the average probability density within bin $j$. Rearrangement gives the exact finite-bin identityFor a continuous probability density on $[0,1]$, uniform continuity makes the bin averages converge uniformly to the density. Since $t\ln t$ is continuous at zero with the stated convention, the right-hand side converges to $h_e(X)$. Thus the increasing resolution cost is $\ln n$; differential entropy is the remaining offset, rather than the entire discrete encoding cost.

For a continuous uniform distribution, every bin has probability $1/n$, and $H_e(Q_n)=\ln n$ exactly. For the continuous uniform distribution's squared transformation above, the exact bin probabilities areThe diagram computes these masses directly, including the bin next to the integrable singularity. It compares their corrected information entropy with the analytic differential entropy $\ln 2-1$.

This toy image compression method stores one rounded arithmetic mean for each disjoint $2\times 2$ block of grayscale pixels. The quantization rule is round to the nearest integer, with exact halves rounded upward. Each stored value is an unsigned eight-bit integer. Reconstruction fills the original block with that value, a form of nearest-neighbor interpolation. Averaging already discards information; rounding can discard more.

Here is an exact $16\times 16$ source array. Index rows and columns by $r,c\in \{0,\dots ,15\}$ and defineThe floor function makes $b$ constant within each block; the indicator function adds 1 only to its upper-left pixel. All values are integers from 20 to 215. Each block therefore has the exact formFor example, the upper-left block $\left(\begin{array}{cc}61& 20\\ 20& 60\end{array}\right)$ has arithmetic mean $161/4=40.25$ and is stored as 40. The block immediately to its right $\left(\begin{array}{cc}73& 32\\ 32& 72\end{array}\right)$ is stored as 52. The complete stored $8\times 8$ array is specified by $q_{ij}=40+10i+12j$ for $0\le i,j\le 7$. Its first row is $40,52,64,76,88,100,112,124$; its last is $110,122,134,146,158,170,182,194$.

This toy file format again has eight-bit width and height fields followed by a one-bit mode: 0 means raw eight-bit grayscale pixels, 1 means the rounded block arithmetic means. This profile accepts only width and height 16. The decoder knows the block size and reconstruction rule from the format, reads the 64 stored values in row order, and repeats each across its corresponding $2\times 2$ block. The original 256 pixels cost $256\cdot 8=2048$ payload bits; the 64 stored values cost $64\cdot 8=512$ payload bits. The reduction is $75\%$. These payload counts exclude the common 17-bit file header; complete sizes are 2065 and 529 bits, or 2072 and 536 bits after final zero padding to eight-bit units. There is no stored codebook or additional prediction rule.

The empirical mean squared error averages squared differences over the 256 original pixel positions:Every block has the same four errors, so this value is exact in squared stored shade units. If the unrounded arithmetic mean $b+1/4$ could be reconstructed, the error would instead be $410.1875$; eight-bit quantization adds $0.0625$. That hypothetical comparison does not count fractional means as free eight-bit values.

The broad shade pattern survives, while the alternating bright and dark pixels vanish. Indeed, a second source with every block constant at $b$ produces the identical stored array $q$: exact decoding of both sources is impossible. This makes the storage benefit and the lost detail visible at once. Block averaging is useful for a coarse preview when the individual pixels are dispensable; choosing an acceptable mean squared error or visual result is part of the application, not a guarantee supplied by the encoder.

A pixel is one picture element at an indexed position in a raster image. It can contain a single grayscale value or several colour components. The word can also refer to a physical display element; the indexed element of the stored image is the meaning used here. One stored pixel need not occupy exactly one physical display element when the image is resized. Microsoft's bitmap documentation likewise distinguishes an image's grid and its numerical colour representation.^[ref]

Separate position from value: raster sampling and reconstruction concerns the spatial grid, while sample depth concerns how many bits encode each component. Four times as many pixels can describe a finer grid without improving the quantization of any value. Conversely, more bits per sample can distinguish more shades without locating an edge more precisely.

In the binary example under run-length encoding, one bit per pixel selects black or white. In the grayscale example under lossy block averaging, eight bits per pixel select one of 256 codes. Calling them codes keeps their numerical values distinct from the physical light that a display emits.

Quantization maps numerical values to a specified discrete collection of representable values. In lossy block averaging, $Q(m)=\lfloor m+1/2\rfloor$ maps block arithmetic means in $[0,255]$ to eight-bit integers. Thus $Q(40.25)=40$ and $Q(40.5)=41$. Different means can share a stored value, making this step irreversible.

A point-sampling model assigns $p_{rc}=f(x_c,y_r)$ to a pixel on a grid, where $f$ is a continuous function representing an image signal. An area-sampling model instead assigns the average value of a function over a specified footprint. These are different models: a narrow feature between point samples can be missed, whereas its contribution to an area sample depends on its width and shade. Neither model says that the recorded value uniquely determines everything inside a grid cell.

The upper panels start with exactly the same $6\times 4$ array. Nearest-neighbor interpolation repeats its values into a $24\times 16$ display; successive linear interpolations in the two coordinate directions produce a smoother $24\times 16$ display. Original sample positions are marked. Both displays use 384 output pixels, but only the same 24 recorded values. The extra values follow the chosen interpolation rule. Between-grid appearance is inferred, not independently recorded.

Spatial resolution needs a stated meaning. A grid over a rectangle of width $L_x$ and height $L_y$ has nominal cell pitches $\Delta x=L_x/W$ and $\Delta y=L_y/H$. Over the same rectangle, doubling both dimensions halves these pitches and quadruples the pixel count. That is a finer grid, not a guarantee of more recoverable detail. Blurring before sampling, quantization and aliasing can make different originals share the same recorded array. Enlarging the rectangle without adding samples instead increases the displayed pitch.

A reproducible computational demonstration makes the ambiguity exact. Take the one-dimensional profiles $f(x)=0.5+0.4\sin (2\pi x)$ and $g(x)=0.5+0.4\sin (10\pi x)$, repeated identically along every row. Sample at $x_j=j/4$ for $j=0,\dots ,8$. Since $10\pi x_j=2\pi x_j+2\pi j$, the fact that the sine is a periodic function gives $f(x_j)=g(x_j)$. Both yield $0.5,0.9,0.5,0.1,0.5,0.9,0.5,0.1,0.5$, although at $x=1/8$ they give approximately $0.78284$ and $0.21716$. The lower panel draws both originals and their common samples: no operation on those samples alone can identify the original profile. This is aliasing in an ideal point-sampling model, without image noise or rounding.

The Nyquist–Shannon sampling theorem describes reconstruction under a bandwidth restriction and ideal sampling assumptions; it does not grant uniqueness for arbitrary image signals from a finite raster image.^[ref] The beautiful lesson is constructive: choose what detail the grid should preserve before sampling, and choose a reconstruction rule suited to the information actually retained.

The sample depth $b$ is the number of bits allocated to one stored sample. An unsigned $b$-bit integer has $2^b$ codes, $0,\dots ,2^b-1$. This is per component, not automatically per pixel: three eight-bit components require 24 payload bits per pixel. A raster image with $WH$ pixels and $k$ such components requires $WHkb$ payload bits before data compression, padding or a file header.

For a toy normalized shade $x\in [0,1]$, define uniform quantization with endpoints included byThe floor function implements rounding to the nearest code, with exact halves rounded upward. The reconstruction step is ${\delta }_b=1/(2^b-1)$, and $|{\hat{x}}_b-x|\le {\delta }_b/2$. For $b=2$, the levels are $0,1/3,2/3,1$; for $b=8$ they are $0,1/255,\dots ,1$. Increasing sample depth refines this value grid without changing the spatial grid. These levels are uniform in $x$; if $x$ is encoded sRGB, they are not uniform in relative luminance.

PNG uses a bit-depth field per sample or palette index. Its grayscale type without alpha permits 1, 2, 4, 8 or 16 bits per sample; its truecolour type permits 8 or 16 bits per component. These are format-specific restrictions, not a rule for every raster image.^[ref]