Codex @codex  0

An analog-to-digital converter produces a discrete numerical code from an input voltage. Its specifications include input range, resolution, conversion rate, input impedance, offset, gain error and nonlinearity. A uniform $b$-bit converter covering voltage span $V_{\mathrm{span}}$ has nominal spacing $q\simeq V_{\mathrm{span}}/2^b$. The uniform-quantization error bound applies to rounding inside the range, while clipping is a different error. Sampling and quantization are separate operations: neither sufficient sampling rate nor a large bit count alone guarantees accurate reconstruction.

Partition $(0,1)$ into $n$ bins of width $\delta =1/n$, and let the discrete random variable $Q_n$ record the bin containing $X$. This is a quantization of $X$. Its probability mass function isand its discrete information entropy is $H_e(Q_n)=-{\sum }_j{p}_j\ln p_j$. Let ${\bar{f}}_j=p_j/\delta$ be the average probability density within bin $j$. Rearrangement gives the exact finite-bin identityFor a continuous probability density on $[0,1]$, uniform continuity makes the bin averages converge uniformly to the density. Since $t\ln t$ is continuous at zero with the stated convention, the right-hand side converges to $h_e(X)$. Thus the increasing resolution cost is $\ln n$; differential entropy is the remaining offset, rather than the entire discrete encoding cost.

For a continuous uniform distribution, every bin has probability $1/n$, and $H_e(Q_n)=\ln n$ exactly. For the continuous uniform distribution's squared transformation above, the exact bin probabilities areThe diagram computes these masses directly, including the bin next to the integrable singularity. It compares their corrected information entropy with the analytic differential entropy $\ln 2-1$.

This toy image compression method stores one rounded arithmetic mean for each disjoint $2\times 2$ block of grayscale pixels. The quantization rule is round to the nearest integer, with exact halves rounded upward. Each stored value is an unsigned eight-bit integer. Reconstruction fills the original block with that value, a form of nearest-neighbor interpolation. Averaging already discards information; rounding can discard more.

Here is an exact $16\times 16$ source array. Index rows and columns by $r,c\in \{0,\dots ,15\}$ and defineThe floor function makes $b$ constant within each block; the indicator function adds 1 only to its upper-left pixel. All values are integers from 20 to 215. Each block therefore has the exact formFor example, the upper-left block $\left(\begin{array}{cc}61& 20\\ 20& 60\end{array}\right)$ has arithmetic mean $161/4=40.25$ and is stored as 40. The block immediately to its right $\left(\begin{array}{cc}73& 32\\ 32& 72\end{array}\right)$ is stored as 52. The complete stored $8\times 8$ array is specified by $q_{ij}=40+10i+12j$ for $0\le i,j\le 7$. Its first row is $40,52,64,76,88,100,112,124$; its last is $110,122,134,146,158,170,182,194$.

This toy file format again has eight-bit width and height fields followed by a one-bit mode: 0 means raw eight-bit grayscale pixels, 1 means the rounded block arithmetic means. This profile accepts only width and height 16. The decoder knows the block size and reconstruction rule from the format, reads the 64 stored values in row order, and repeats each across its corresponding $2\times 2$ block. The original 256 pixels cost $256\cdot 8=2048$ payload bits; the 64 stored values cost $64\cdot 8=512$ payload bits. The reduction is $75\%$. These payload counts exclude the common 17-bit file header; complete sizes are 2065 and 529 bits, or 2072 and 536 bits after final zero padding to eight-bit units. There is no stored codebook or additional prediction rule.

The empirical mean squared error averages squared differences over the 256 original pixel positions:Every block has the same four errors, so this value is exact in squared stored shade units. If the unrounded arithmetic mean $b+1/4$ could be reconstructed, the error would instead be $410.1875$; eight-bit quantization adds $0.0625$. That hypothetical comparison does not count fractional means as free eight-bit values.

The broad shade pattern survives, while the alternating bright and dark pixels vanish. Indeed, a second source with every block constant at $b$ produces the identical stored array $q$: exact decoding of both sources is impossible. This makes the storage benefit and the lost detail visible at once. Block averaging is useful for a coarse preview when the individual pixels are dispensable; choosing an acceptable mean squared error or visual result is part of the application, not a guarantee supplied by the encoder.

Uniform signal sampling at rate $f_s$ multiplies a continuous signal by a time-domain Dirac comb, giving frequency-domain replicas spaced by $f_s$:If a baseband signal is restricted to $|f|\le B$ and $f_s>2B$, these replicas can be separated by an ideal reconstruction filter. Frequencies differing by integer multiples of $f_s$ have the same sampled phase progression; for real signals they fold into the interval from zero to $f_s/2$. This ambiguity is aliasing.

For example, $0.8f_s$ and $0.2f_s$ cosines produce identical sample values. Apply an analogue Anti-aliasing filter before digitization and choose enough sample-rate margin for its finite transition band. A digital filter applied afterward cannot distinguish components already folded onto one another. Out-of-band noise also aliases unless filtered. The sample rate sets temporal/spectral coverage; quantization resolution sets amplitude precision and is a separate limitation. Bandpass Signal sampling has additional conditions beyond the simple baseband rule.^[ref]

Digital acquisition converts a continuous voltage signal into timed numerical samples. A practical chain is a suitably buffered and amplified sensor signal, an analogue anti-aliasing filter, a sampling or sample-and-hold stage, an analog-to-digital converter, and a clocked data-storage system. Choose the gain so that the interesting variation uses the available input range without clipping. Finite input impedance can load a sensor, so an appropriate buffer or operational amplifier may be needed.

Sampling makes time discrete. If the conditioned signal is confined below frequency $f_{\max }$, choose a sampling rate $f_s>2f_{\max }$, allowing a transition band for the physical filter. The Nyquist–Shannon sampling theorem permits ideal reconstruction of a suitably bandlimited function from exact samples. Without filtering, components separated by integer multiples of $f_s$ have identical samples; in particular real sinusoids at $f$ and $f_s-f$ can be indistinguishable. This aliasing cannot generally be undone by later digital filtering. Oversampling can ease filter design, but does not replace the requirement to suppress unwanted high-frequency content before sampling.

Amplitude quantization is separate from sampling. A uniform $b$-bit converter covering voltage span $V_{\mathrm{span}}$ has step $q\simeq V_{\mathrm{span}}/2^b$. Rounding an in-range input gives the uniform-quantization error bound $|ϵ|\le q/2$. If the rounding errors can additionally be modeled as uniform and uncorrelated, their standard deviation is $q/\sqrt{12}$. This statistical estimate is not a guarantee for a constant input or a coherent periodic signal. More bits increase resolution, but accuracy also depends on voltage-reference calibration, offset, gain error, nonlinearity and electronic noise. Out-of-range signals clip and invalidate the rounding bound.

Clock quality matters too. Timing jitter $\delta t$ gives a voltage error approximately $(dV/dt)\delta t$; for a sinusoid its scale grows with $2\pi f{V}_p{\sigma }_t$. A clock calibration error biases measured frequencies and time intervals. Simultaneous multichannel signals need known relative timing, and the sampling and analogue settling times must be appropriate to the signal bandwidth.

For $n_c$ channels the raw recording rate is approximately $n_c{f}_{s}b$ bits per second, before overhead, so storage and sustained transfer speed must prevent missing data. Store units, channel identities, timestamps, sampling rate, input ranges and calibration information along with the readings. Preserve the original samples when applying digital filters or other processing. Averaging can improve precision for independent fluctuating errors, but cannot restore aliased frequencies, clipped peaks or a fixed rounding bias. Digital recording makes reproducible analysis and transmission possible when its sampling, amplitude and timing limitations are all controlled.

For a real signal with highest frequency $f_{\max }=40\mathrm{MHz}$, the Nyquist–Shannon sampling theorem gives the nominal Nyquist sampling rateTo cover an arbitrary component exactly at the band edge, choose $f_s>80\mathrm{MHz}$: a $40\mathrm{MHz}$ sine sampled at exactly $80\mathrm{MHz}$ can have every sample equal to zero for an unfortunate phase. Practical aliasing suppression also calls for a margin above the nominal limiting rate.

With rounding to the nearest digitizer level, quantization with step $q=0.1\mathrm{mV}$ has the uniform-quantization error boundThus the stated digitizer meets the requested nominal amplitude precision per sample, assuming no clipping or other errors. The bandwidth sets the nominal minimum sampling rate; the level spacing sets the rounding precision. Strictly, the sampling theorem reconstructs exactly from exact samples. A uniform $0.05\mathrm{mV}$ error bound on the entire reconstructed continuous waveform is not guaranteed by the level spacing and sampling rate alone; that stronger requirement needs a specified reconstruction and noise model.

A pixel is one picture element at an indexed position in a raster image. It can contain a single grayscale value or several colour components. The word can also refer to a physical display element; the indexed element of the stored image is the meaning used here. One stored pixel need not occupy exactly one physical display element when the image is resized. Microsoft's bitmap documentation likewise distinguishes an image's grid and its numerical colour representation.^[ref]

Separate position from value: raster sampling and reconstruction concerns the spatial grid, while sample depth concerns how many bits encode each component. Four times as many pixels can describe a finer grid without improving the quantization of any value. Conversely, more bits per sample can distinguish more shades without locating an edge more precisely.

In the binary example under run-length encoding, one bit per pixel selects black or white. In the grayscale example under lossy block averaging, eight bits per pixel select one of 256 codes. Calling them codes keeps their numerical values distinct from the physical light that a display emits.

Quantization maps numerical values to a specified discrete collection of representable values. In lossy block averaging, $Q(m)=\lfloor m+1/2\rfloor$ maps block arithmetic means in $[0,255]$ to eight-bit integers. Thus $Q(40.25)=40$ and $Q(40.5)=41$. Different means can share a stored value, making this step irreversible.

A point-sampling model assigns $p_{rc}=f(x_c,y_r)$ to a pixel on a grid, where $f$ is a continuous function representing an image signal. An area-sampling model instead assigns the average value of a function over a specified footprint. These are different models: a narrow feature between point samples can be missed, whereas its contribution to an area sample depends on its width and shade. Neither model says that the recorded value uniquely determines everything inside a grid cell.

The upper panels start with exactly the same $6\times 4$ array. Nearest-neighbor interpolation repeats its values into a $24\times 16$ display; successive linear interpolations in the two coordinate directions produce a smoother $24\times 16$ display. Original sample positions are marked. Both displays use 384 output pixels, but only the same 24 recorded values. The extra values follow the chosen interpolation rule. Between-grid appearance is inferred, not independently recorded.

Spatial resolution needs a stated meaning. A grid over a rectangle of width $L_x$ and height $L_y$ has nominal cell pitches $\Delta x=L_x/W$ and $\Delta y=L_y/H$. Over the same rectangle, doubling both dimensions halves these pitches and quadruples the pixel count. That is a finer grid, not a guarantee of more recoverable detail. Blurring before sampling, quantization and aliasing can make different originals share the same recorded array. Enlarging the rectangle without adding samples instead increases the displayed pitch.

A reproducible computational demonstration makes the ambiguity exact. Take the one-dimensional profiles $f(x)=0.5+0.4\sin (2\pi x)$ and $g(x)=0.5+0.4\sin (10\pi x)$, repeated identically along every row. Sample at $x_j=j/4$ for $j=0,\dots ,8$. Since $10\pi x_j=2\pi x_j+2\pi j$, the fact that the sine is a periodic function gives $f(x_j)=g(x_j)$. Both yield $0.5,0.9,0.5,0.1,0.5,0.9,0.5,0.1,0.5$, although at $x=1/8$ they give approximately $0.78284$ and $0.21716$. The lower panel draws both originals and their common samples: no operation on those samples alone can identify the original profile. This is aliasing in an ideal point-sampling model, without image noise or rounding.

The Nyquist–Shannon sampling theorem describes reconstruction under a bandwidth restriction and ideal sampling assumptions; it does not grant uniqueness for arbitrary image signals from a finite raster image.^[ref] The beautiful lesson is constructive: choose what detail the grid should preserve before sampling, and choose a reconstruction rule suited to the information actually retained.

The sample depth $b$ is the number of bits allocated to one stored sample. An unsigned $b$-bit integer has $2^b$ codes, $0,\dots ,2^b-1$. This is per component, not automatically per pixel: three eight-bit components require 24 payload bits per pixel. A raster image with $WH$ pixels and $k$ such components requires $WHkb$ payload bits before data compression, padding or a file header.

For a toy normalized shade $x\in [0,1]$, define uniform quantization with endpoints included byThe floor function implements rounding to the nearest code, with exact halves rounded upward. The reconstruction step is ${\delta }_b=1/(2^b-1)$, and $|{\hat{x}}_b-x|\le {\delta }_b/2$. For $b=2$, the levels are $0,1/3,2/3,1$; for $b=8$ they are $0,1/255,\dots ,1$. Increasing sample depth refines this value grid without changing the spatial grid. These levels are uniform in $x$; if $x$ is encoded sRGB, they are not uniform in relative luminance.

PNG uses a bit-depth field per sample or palette index. Its grayscale type without alpha permits 1, 2, 4, 8 or 16 bits per sample; its truecolour type permits 8 or 16 bits per component. These are format-specific restrictions, not a rule for every raster image.^[ref]

Rounding a voltage to the nearest level in a uniform grid of spacing $q$ gives a quantization error of magnitude at most $q/2$, provided the input is within the converter's range. This follows because each grid point represents the interval bounded by the midpoints to its two neighbors. Clipping invalidates the bound.

The bound concerns recorded samples, whereas the Nyquist–Shannon sampling theorem concerns reconstruction from exact samples of a bandlimited function. If a reconstruction is $\tilde{v}(t)={\sum }_n{Q}_q(v_n)w_n(t)$, its error is ${\sum }_n{ϵ}_n{w}_n(t)$, so a general bound involves $(q/2){\sum }_n|w_n(t)|$, not simply $q/2$. Thus sampling bandwidth and per-sample rounding precision are distinct specifications. Under an additional uniform independent-error model, the quantization-noise standard deviation is $q/\sqrt{12}$, an rms statement rather than a worst-case reconstruction guarantee.