Image processing

Image processing transforms recorded image signal data to remove noise, recover detail or extract structure. Variational image processing describes desired reconstructions through energies, while diffusion image processing describes filtering through evolution equations.

Quantization maps numerical values to a specified discrete collection of representable values. In lossy block averaging, $Q(m)=\lfloor m+1/2\rfloor$ maps block arithmetic means in $[0,255]$ to eight-bit integers. Thus $Q(40.25)=40$ and $Q(40.5)=41$. Different means can share a stored value, making this step irreversible.

Rounding a voltage to the nearest level in a uniform grid of spacing $q$ gives a quantization error of magnitude at most $q/2$, provided the input is within the converter's range. This follows because each grid point represents the interval bounded by the midpoints to its two neighbors. Clipping invalidates the bound.

The bound concerns recorded samples, whereas the Nyquist–Shannon sampling theorem concerns reconstruction from exact samples of a bandlimited function. If a reconstruction is $\tilde{v}(t)={\sum }_n{Q}_q(v_n)w_n(t)$, its error is ${\sum }_n{ϵ}_n{w}_n(t)$, so a general bound involves $(q/2){\sum }_n|w_n(t)|$, not simply $q/2$. Thus sampling bandwidth and per-sample rounding precision are distinct specifications. Under an additional uniform independent-error model, the quantization-noise standard deviation is $q/\sqrt{12}$, an rms statement rather than a worst-case reconstruction guarantee.

Nearest-neighbor interpolation assigns each requested location the value of its nearest available sample, with a specified convention for ties. In lossy block averaging, samples are placed at block centres and each is copied into its four original pixel positions. This enlarges the raster image without recovering discarded detail.

Image segmentation divides an image signal domain into regions according to a chosen criterion, such as approximately constant or smoothly varying intensity. The Mumford–Shah segmentation model balances fidelity, within-region smoothness and image edge length. An intensity segmentation need not coincide with semantic object identities.

A grey-value image signal is a scalar function on an image signal domain or an array of pixel values. Continuous image processing models use spaces such as $L^2$, Sobolev spaces or the BV space, while discrete models use finite-dimensional arrays. The chosen representation distinguishes smooth transitions, noise and image edges.

Image noise is unwanted variation in an observed image signal. In an additive model $g=u+\eta$, Gaussian noise motivates a squared data-fidelity penalty, while other statistics require different penalties. Image smoothing suppresses rapid fluctuations, but a filter must distinguish noise from genuine image edges.

For a smoothed image signal $u_{\sigma }$, average the gradient outer product as $J_{\rho }=G_{\rho }\ast (\nabla u_{\sigma }\nabla u_{\sigma }^T)$. The dominant eigenvector estimates the normal to a coherent image edge, and eigenvalue contrast measures local directional structure. Two distinct scales separate differentiation from orientation averaging.

An image edge is a rapid grey-value transition, modeled as a large gradient or a jump discontinuity. It differs from an edge of a graph. Variational models can penalize image edge length, while diffusion image processing estimates image edge directions to limit image smoothing across them.

image edge enhancement increases the apparent sharpness or contrast of an image edge. Unsharp masking gives bounded linear high-frequency amplification; backward normal diffusion in the Perona-Malik equation gives formal nonlinear sharpening with ill-posedness risks. Positive diffusion tensors chiefly preserve or connect structure rather than perform unbounded backward diffusion.

The formal shock-filter equation $u_t=-\mathrm{sign}(\Delta u)|\nabla u|$ steepens transitions around inflection boundaries. It is a Hamilton–Jacobi-type transport mechanism rather than positive diffusion. Combining it with controlled forward image smoothing limits noise amplification.

Subtract a smoothed image signal and add a scaled residual: $u=g+\lambda (g-G_t\ast g)$. The Fourier multiplier $1+\lambda (1-e^{-t|\xi {|}^2})$ is bounded by $1+\lambda$. This enhances apparent contrast but also amplifies noise; it is not a stable exact inverse of heat image smoothing.

Diffusion filtering evolves an observed image signal using a diffusion equation. The heat equation gives Gaussian image smoothing, while gradient-dependent or tensor-dependent diffusion can reduce transport across image edges. A image smoothing scale, stopping rule and appropriate boundary conditions are part of the filter.

A positive tensor $D=a_n{e}_n{e}_n^T+a_t{e}_t{e}_t^T$ selects different diffusion strengths normal and tangent to an estimated image edge. Taking $0<a_n\ll a_t$ smooths along the image edge while reducing mixing across it. A structure tensor provides robust orientation estimates. Positive eigenvalues ensure forward local parabolicity.

Image smoothing suppresses rapid fluctuations attributed to noise. Linear heat equation image smoothing attenuates Fourier frequencies by $e^{-t|\xi {|}^2}$ but also blurs image edges. Nonlinear filtering uses image signal geometry to distinguish fluctuations from meaningful transitions.

A median filter replaces an image signal value by the median of nearby values. Unlike a linear average, it can reject isolated extreme observations without averaging both sides of an image edge. The small-radius response depends on neighbourhood geometry and measure; a uniform disk-area median and a circle-boundary median have different time scales.

At a smooth point with nonzero gradient, rotate coordinates so $\nabla u=G{e}_2$. Write $u=u_0+Gy+\frac{1}{2}(A{x}^2+2Bxy+C{y}^2)+O(|(x,y){|}^3)$. A candidate median $u_0+d{h}^2$ has level boundary $y=(d{h}^2-A{x}^2/2)/G+O(h^3)$ in the disk. The area imbalance is $(2d-A/3)h^3/G+o(h^3)$; equal half-areas force $d=A/6$. Here $A=\Delta u-\nabla u^T(D^{2}u)\nabla u/|\nabla u{|}^2$ is the tangential second derivative, giving the formula. The disk uses uniform area measure, not its boundary measure.

Gaussian blur applies convolution with $G_{\sigma }(x)=(2\pi {\sigma }^2{)}^{-n/2}{e}^{-|x{|}^2/(2{\sigma }^2)}$ to an image signal. It averages over a spatial scale $\sigma$, suppressing rapid variation while also blurring image edges. The heat-kernel convolution realizes this filter at time $t={\sigma }^2/2$.

The Fourier transform of a Gaussian and the convolution theorem give this multiplier with the angular-frequency convention $e^{-ix\cdot \xi }$. In cycles-per-length frequencies $k$, it is $e^{-2{\pi }^2{\sigma }^2|k{|}^2}$. Each coordinate has variance ${\sigma }^2$, and higher frequencies receive stronger attenuation.

A variational model balances agreement with observed data against a regularity or geometric penalty. Examples include total variation denoising, the relaxed graph-area functional, and the Mumford–Shah functional. Convex penalties support unique reconstruction, while segmentation energies can have multiple competing partitions.

The energy ${\int }_{\Omega \setminus K}(u-g{)}^2+\alpha {\int }_{\Omega \setminus K}|\nabla u{|}^2+\beta {\mathcal{H}}^1(K)$ balances data fidelity, within-region smoothness and image edge length. In its relaxed SBV space formulation the image edge set is $J_u$. Clipping to the bounded data range, the SBV compactness theorem and lower semicontinuity prove existence; the full segmentation problem is not strictly convex.

The Mumford–Shah functional can be approximated by an auxiliary edge field $0\le v\le 1$ and energyThe field is near one inside regions and near zero across an image edge, weakening smoothing there. The profile $v(s)=1-e^{-|s|/(2\epsilon )}$ has unit edge energy per unit length in this normalization. The 1990 construction is a Gamma-convergence approximation; its global-minimizer limit does not guarantee global optimality of an alternating numerical solve. Each fixed-field subproblem is quadratic, but the joint energy is nonconvex.

Removing the jump-length cost from the Mumford–Shah functional lets finer piecewise-constant partitions reduce squared fidelity without charging for their proliferating image edges. For continuous data on a compact image domain, fine-cell averages converge in squared error while the ordinary within-cell gradient is zero. A finite interface penalty makes that geometry costly.

On the unit square, let $g=0$ and take $u_0={\chi }_{\{x_1>0.4\}}$, $u_1={\chi }_{\{x_1>0.6\}}$. Both have one unit-length jump and zero ordinary gradient. Their mean has two unit-length jumps. For squared-error coefficient one, $[E(u_0)+E(u_1)]/2=0.5+\beta$ while $E((u_0+u_1)/2)=0.45+2\beta$, violating convexity when $\beta >0.05$. The Mumford–Shah functional can therefore be nonconvex even though the fixed-edge reconstruction subproblem is strictly convex.

For a fixed sufficiently regular image edge set, the Mumford–Shah functional is a strictly convex quadratic energy on the Sobolev space of its complement. A first variation gives $u-\alpha \Delta u=g$ within each region and separate one-sided Neumann boundary conditions on free edges. A jump need not vanish across an edge. The weak solution is unique by the Lax-Milgram theorem; this does not prove uniqueness when the edge set is also optimized.

As the jump-length weight tends to infinity with the gradient weight fixed, the limiting problem minimizes $\Vert u-g{\Vert }_2^2+\alpha \Vert \nabla u{\Vert }_2^2$ on $H^1$. It has a unique minimizer by the Lax-Milgram theorem and strict convexity. Without prescribed boundary values, its weak equation is $u-\alpha \Delta u=g$ with natural Neumann boundary conditions.

The infinite gradient-weight limit restricts to SBV space image signals with $\nabla u=0$ almost everywhere. Their energy is fidelity plus jump length, represented by constants on a Caccioppoli partition. Internal perimeter is counted once by $\frac{1}{2}{\sum }_i\mathrm{Per}(E_i;\Omega )$; adjacent equal-valued regions can be merged.

Suppose two constant data intensities differ by $d$, have areas $a$ and $A-a$, and share an internal boundary of length $L$. Keeping the boundary costs $\beta L$ with zero fidelity; merging the two regions costs $a(A-a)d^2/A$ after fitting their common mean. The displayed inequality makes splitting cheaper among these two candidate geometries. This comparison does not prove either geometry is globally optimal among all partitions.

When grey values are fixed but region labels vary, a smooth interface satisfies $\beta \kappa =(c_j-g{)}^2-(c_i-g{)}^2$. The curvature uses the normal pointing out of region $i$, positive on an outward-oriented circle. It balances the change in data cost against the first variation of interface length. Fixing a full spatial image signal instead fixes its jumps and leaves no such boundary-relocation freedom.

At a freely moving regular junction of three smooth image edges with equal isotropic length weights, the endpoint first variation requires the sum of their outward unit tangent vectors to vanish. Three unit vectors with zero sum have pairwise angles of $120$ degrees. Different interface weights instead give a weighted force balance, and constrained or singular junctions require separate hypotheses.

For a fixed positive-area region $E_i$, its least-squares grey value is $c_i=|E_i{|}^{-1}{\int }_{E_i}g$. The minimum fidelity equals $\int g^2-{\sum }_i({\int }_{E_i}g{)}^2/|E_i|$. This eliminates the grey values before optimizing the partition geometry.

A minimizer of the Mumford–Shah functional with bounded image signal data has an essentially closed jump set: replacing $J_u$ by its relative closure adds no $(n-1)$-dimensional measure. Its complement supports a Sobolev representative. This regularity theorem connects a relaxed SBV space minimizer to the classical closed-edge formulation; it is not a property of every special bounded-variation function.