Quantization (signal processing, $Q(x)$)

Quantization maps numerical values to a specified discrete collection of representable values. In lossy block averaging, $Q(m)=\lfloor m+1/2\rfloor$ maps block arithmetic means in $[0,255]$ to eight-bit integers. Thus $Q(40.25)=40$ and $Q(40.5)=41$. Different means can share a stored value, making this step irreversible.

Rounding a voltage to the nearest level in a uniform grid of spacing $q$ gives a quantization error of magnitude at most $q/2$, provided the input is within the converter's range. This follows because each grid point represents the interval bounded by the midpoints to its two neighbors. Clipping invalidates the bound.

The bound concerns recorded samples, whereas the Nyquist–Shannon sampling theorem concerns reconstruction from exact samples of a bandlimited function. If a reconstruction is $\tilde{v}(t)={\sum }_n{Q}_q(v_n)w_n(t)$, its error is ${\sum }_n{ϵ}_n{w}_n(t)$, so a general bound involves $(q/2){\sum }_n|w_n(t)|$, not simply $q/2$. Thus sampling bandwidth and per-sample rounding precision are distinct specifications. Under an additional uniform independent-error model, the quantization-noise standard deviation is $q/\sqrt{12}$, an rms statement rather than a worst-case reconstruction guarantee.