Inverse problem ($A(x)=y$)

An inverse problem reconstructs an unknown object $x$ from observed data $y$ and a specified model $A(x)=y$. The associated forward problem predicts the observations from $x$. In image deblurring, $x$ contains sharp pixel values and a matrix $A$ mixes neighbouring pixels; in tomography, the observations are integrals through the object. The appeal is that indirect measurements can reveal otherwise inaccessible structure. The difficulty is that the forward problem may erase precisely the distinctions we want to recover.

Consider this completely computable two-pixel image deblurring model:Each measured pixel retains $55\%$ of its own value and receives $45\%$ of its neighbour's. Adding measurement noise $e=(0.01,-0.01{)}^T$ gives $y^{\delta }=(0.56,0.44{)}^T$, whose direct reconstruction is $A^{-1}{y}^{\delta }=(1.1,-0.1{)}^T$. The Euclidean norm of the reconstruction error is $0.1\sqrt{2}$, ten times the data error $0.01\sqrt{2}$. A negative second pixel appears even though both observed pixels are positive.

Despite this wrong reconstruction, its residual of an inverse problem is exactly zero: the predicted data equal the noisy observations. Thus matching the data perfectly does not certify reconstruction accuracy. The relevant question is how large a change in the unknown could hide within the measurement uncertainty. A condition number quantifies that sensitivity for an invertible matrix; the null space identifies changes that no improvement in precision can reveal.

The exact singular value decomposition explains this amplification. With the orthonormal basisthe mean mode survives unchanged while the difference mode shrinks tenfold. Direct inversion multiplies difference-mode measurement noise by ten. The condition number is ${\kappa }_2(A)={\sigma }_{\max }/{\sigma }_{\min }=10$. The beauty of the singular value decomposition is this separation into independent channels: it identifies which information survives and which needs protection by regularization of an inverse problem.

A Riemannian metric is boundary rigid within a specified class if the distances it induces between all pairs of boundary points determine the interior metric, up to a diffeomorphism that fixes the boundary pointwise. The qualification is unavoidable: relabeling interior points in this way changes the coordinate expression of the metric without changing any boundary distance.

This inverse problem idealizes recovering an interior wave-speed geometry from boundary travel times. Its data are shortest-path distances, so arbitrary trapped regions need not be visible. For a Euclidean disk, multiplying the metric by $c^2$ multiplies every boundary distance by $c$; this simple experiment already distinguishes a uniform change of travel-time scale. Determining a general spatially varying metric is much harder. A landmark theorem proves boundary rigidity for compact simple two-dimensional Riemannian manifolds, where the boundary is strictly convex and the relevant geodesics have no conjugate-point ambiguity.^[ref]

Different subsurface models can produce the same measured field. A gravity anomaly, for example, does not uniquely separate density, body shape and regional background. Geological constraints and complementary observations reduce this ambiguity; an inferred density should retain the model and reference-field assumptions used to obtain it.

Inference of subsurface geophysical density contrast from a gravity anomaly. Different geometries and density distributions can produce the same external field, so independent geological or seismic constraints are essential.

Image registration estimates a coordinate transformation aligning corresponding image content. If $I$ is the moving image and $J$ is the reference, one simple model chooses a transformation $T$ to minimizeThe first term compares image values; $R$ penalizes unwanted deformation. A translation model $T(x)=x-a$ has only a displacement vector to estimate, while a spatially varying deformation needs many more parameters and stronger regularization.

The transformation direction is part of the convention: this expression samples the moving image at $T(x)$ for each reference location. The sampling requires interpolation when $T(x)$ falls between pixel centers. Boundary handling must also specify what happens outside the image domain.

Intensity agreement does not prove correct correspondence. Repeated patterns can produce several alignment minima, and a change of imaging modality can alter intensities even at matching physical locations. The transformation family, similarity objective, interpolation rule and deformation penalty are distinct modeling decisions in this inverse problem.

Phase retrieval reconstructs a signal from measurements that discard its complex wave phase. A basic diffraction model measures $I(\xi )=|\hat{f}(\xi ){|}^2$, rather than the complex Fourier transform itself. The lost phase cannot generally be recovered simply by taking a square root of the intensity.

There are immediate ambiguities. Multiplication by a constant $e^{i\theta }$ leaves every magnitude unchanged. A translation multiplies the transform by a unit-modulus phase factor, and conjugate reflection $f(x)\mapsto \overline{f(-x)}$ conjugates its transform. These transformations therefore preserve $I$. They need not exhaust the ambiguities of one-dimensional data.^[ref]

By the convolution theorem, magnitude-squared data determine the autocorrelation of a square-integrable function under suitable inversion hypotheses, rather than automatically determining the function itself. Known support, additional masked measurements or other structural assumptions can reduce the ambiguity; their sufficiency must be checked for the actual measurement model. Frequency-resolved optical gating uses a family of delayed nonlinear spectra, giving more information than a single unmodified Fourier magnitude.

Seismic tomography infers a spatial model of elastic-wave propagation from many source-receiver observations. Uneven illumination, finite wavelength, parameter tradeoffs and regularization limit its resolution. A seismic-velocity image is not directly an image of temperature or flow velocity.

Surface-wave tomography combines path-dependent measurements of seismic surface-wave dispersion to infer lateral and depth variation in elastic properties. Joint Rayleigh wave and Love wave measurements can help constrain seismic anisotropy.

Seismic source inversion estimates source location, origin time and source strength or mechanism from seismograms. A symmetric seismic moment tensor has six independent entries. At fixed location and origin time, its predicted seismograms are linear in those entries, allowing variable projection to reduce the nonlinear search.

Image deblurring reconstructs a sharper image from observations described by a blur model. In the linear model, a matrix or linear operator mixes values of the unknown image; measurement noise perturbs the observed result. The inverse problem must therefore recover attenuated distinctions while controlling amplification of measurement noise, as the two-pixel example illustrates.

Tomography reconstructs an interior distribution from measurements along multiple lines or paths through it. A linear unattenuated model is the Radon transform; absorption changes it to an attenuated Radon transform.

Tomography using approximately coincident annihilation-photon detections to constrain an internal emission distribution. A calibrated model accounts for propagation, detector response, unrelated coincidences and scattering.

A pair of nearly simultaneous detector events treated as a candidate two-photon positron annihilation. True unscattered pairs define a coincidence line of response. For attenuation along one such line, the complementary survival path integrals add to the full chord and are independent of the annihilation depth.

The distance between positron emission and annihilation. Its distribution introduces a blur even if the subsequent line of response is measured perfectly.

Coincidence imaging using the detection-time difference to estimate annihilation position along a line of response. For timing difference $\Delta t$, displacement from the midpoint is $c\Delta t/2$.

The line joining two coincident photon detections in ideal two-photon annihilation imaging. Finite detector size, non-collinearity and positron travel limit its relation to the original decay position.

A line defined by an approximately coincident detector pair in annihilation-photon tomography. An event constrains a possible annihilation location to that line within instrumental and physical uncertainties. Time-of-flight data can add a position constraint along it; attenuation, scattering and random coincidences change the forward model.

Transmission measurements determine line integrals of an absorption coefficient after taking the logarithm of incident intensity divided by transmitted intensity. Under the ideal straight-ray model, these measurements are a Radon transform; filtered backprojection reconstructs a planar slice. Angular coverage, calibration and noise control matter because differentiating measured projections amplifies high frequencies. In contrast, single-photon emission computed tomography reconstructs an emitting source through a possibly attenuating medium.

A dimensionless relative attenuation value calibrated so that water is zero and air is approximately $-1000$. It depends on the computed tomography spectrum and material composition. It is not directly a linear attenuation coefficient at the $511\mathrm{keV}$ energy used in positron emission tomography.

X-ray tomography at a scale that can resolve small skeletal features without destroying a specimen. Segmentation and resolution constrain anatomical interpretation.

A form of tomography that reconstructs an internal radioactive source from directional counts of individually emitted photons. Known spatial photon absorption gives the exponential path weight in an attenuated Radon transform. Full directed data and a suitable reconstruction formula recover the source intensity rather than a transmitted exterior-beam attenuation map.

An inverse problem is an ill-posed problem if it fails any requirement of a well-posed problem: existence for every admissible datum, uniqueness, or continuity of the reconstructed object as the data change. These requirements depend on the chosen spaces and norms, not just the equation. Clason's regularization notes formulate this distinction explicitly.^[ref]

Complete averaging illustrates lost information:The difference mode $q_{-}=(1,-1{)}^T/\sqrt{2}$ belongs to the null space because $B{q}_{-}=0$. Even restricting both pixels to $[0,1]$ leaves infinitely many indistinguishable inputs. This is failure of uniqueness. Data such as $(0.51,0.49{)}^T$ also have no exact preimage under $B$, illustrating failure of existence when arbitrary two-component observations are admissible.

The two-pixel matrix $A$ in the inverse problem example has a weakly observed difference mode, not a null space mode. Its inverse is continuous: this finite-dimensional problem is a well-posed problem with noise-sensitive conditioning. Small positive singular values alone do not imply mathematical ill-posedness. In infinite-dimensional problems, positive singular values approaching zero can make inversion unbounded: unit solution modes then produce data tending to zero. This destroys continuity even without a null space. Regularization of an inverse problem must distinguish missing information from information recoverable only at the cost of amplifying measurement noise.

The X-ray transform records integrals of a function or tensor evaluation over a specified family of curves. The curve family, parametrization and any weights are part of the definition. Straight lines, geodesics and magnetic flows give different versions; their injectivity and unavoidable potential kernels depend on that geometry.

The transform integrates a base function and a velocity-evaluated one-form along the closed trajectories of a surface thermostat. Every exact differential form of degree one integrates to zero, because $F(f\circ \pi )=df(v)$. The relevant kernel theorem concerns these functions of velocity degree zero and one, rather than arbitrary functions on the unit tangent bundle.

For a smooth Anosov flow generated by a surface thermostat on a closed oriented Riemannian surface, a smooth base function plus a smooth one-form is a smooth flow coboundary precisely when the base function is zero and the one-form is an exact differential form. This is the nontrivial degree-zero/one kernel theorem. It does not require negative curvature or preservation of Liouville volume. The converse follows directly from $F(f\circ \pi )=df(v)$. A primary proof of the kernel theorem is arxiv.org/abs/math/0601039.

The magnetic transform integrates functions or velocity-evaluated one-forms along magnetic trajectories, rather than ordinary geodesics. For closed trajectories, every exact one-form integrates to zero. Hyperbolic dynamics can make these the entire degree-one kernel.

For a smooth Anosov magnetic flow on a closed oriented surface, a smooth function plus a smooth one-form can be a smooth flow coboundary only when the function is zero and the one-form is exact. Equivalently their sum has zero integral along every closed orbit precisely in that case, using the smooth Livšic theorem. This is the nontrivial magnetic transform kernel theorem. Exact one-forms give the converse directly, since $F(w\circ \pi )=dw(v)$. The theorem does not require pointwise negative Gaussian curvature.

For a symmetric covariant two-tensor, the geodesic transform integrates its evaluation on the unit velocity along boundary-to-boundary or closed geodesics, as appropriate to the manifold. A potential symmetric 2-tensor gives a total derivative along each geodesic and therefore integrates to zero on closed geodesics; boundary data require the corresponding boundary potential to vanish.

The displayed one-form on the unit round sphere has zero integral on every closed geodesic. Indeed, the antipodal map pulls it back to its negative, and the second half of an oriented great circle is the antipodal image of the first. Nevertheless $d\theta =2x_{3}d{x}_3\wedge d{x}_1$ is nonzero on an open set, so the one-form is not a closed differential form and cannot be an exact differential form. Thus a curvature-free extension of one-form injectivity fails.

On a closed oriented Riemannian surface this integral identity follows from the canonical commutators and integration by parts against Liouville volume of a surface geodesic flow. If $VX\phi =-2H\phi$ and $K<0$, substitution gives $\Vert X\phi {\Vert }^2+5\Vert H\phi {\Vert }^2+\int (-K)(V\phi {)}^2=0$. Each term is nonnegative, forcing all three derivatives to vanish. For $\phi =V^{2}u+u$, the fibre equation then makes $u$ a constant plus a fibrewise linear function, supplying the potential symmetric 2-tensor kernel.

For a vector field $Z$, define $\beta (v,w)=⟨{\nabla }_{v}Z,w⟩+⟨{\nabla }_{w}Z,v⟩$. Its diagonal evaluation is $2⟨{\nabla }_{v}Z,v⟩$. Along a geodesic this equals $2d⟨Z,\dot{\gamma }⟩/dt$, so it is invisible to closed-geodesic integrals. If a fibrewise linear function $u(x,v)={\eta }_x(v)$ solves $Xu=\beta (v,v)$, then $Z={\eta }^{♯}/2$ gives this convention.

A forward problem computes observable data $f=Ku$ from a known state or parameter $u$ and a specified model $K$. Its associated inverse problem reconstructs $u$ from $f$ and the model. Inverse problems can fail existence, uniqueness or continuous dependence on the data.

The residual is the difference between predicted and observed data. Its norm measures data misfit and can determine an iteration stopping rule for regularization of an inverse problem.

An inverse scattering problem reconstructs a medium or obstacle from measured scattered waves. Under the Born approximation, fixed-frequency far-field data sample the Fourier transform of the scattering potential on a restricted frequency surface.

For an outgoing field outside a sphere of radius $R$, write $u_s=k{\sum }_{n,m}{i}^{n+1}{a}_n^m{h}_n^{(1)}(kr)Y_n^m$. Its far-field pattern is $\sum a_n^m{Y}_n^m$. If its trace on that sphere lies in L2 space, Parseval's identity gives $k^2\sum |h_n^{(1)}(kR){|}^2|a_n^m{|}^2<\infty$. The fixed-argument growth of spherical Hankel functions yields the displayed equivalent range condition, with the low-order weights defined finitely. Square summability of the far-field coefficients alone does not ensure continuation to the chosen sphere.

Continuation of a far-field pattern to a spherical trace multiplies harmonic coefficients by the displayed factor, whose magnitude grows faster than exponentially with order. A data error of size $\delta$ in one normalized high-order harmonic therefore produces near-field error $\delta k|h_n^{(1)}(kR)|$, unbounded as its order increases. Some noisy square-summable far fields fail the range condition entirely. This proves unstable field continuation; bounds on actual obstacle geometry additionally require an admissible shape class and a stable boundary-extraction procedure.

The Born approximation for scalar wave scattering relates an incident plane wave and the transverse Fourier transform of its scattered field to samples of the three-dimensional Fourier transform of a scattering potential on a shifted Ewald sphere. One incident direction at one frequency samples a surface, rather than an entire three-dimensional frequency region. Additional illuminations, frequencies or structural constraints are needed for a full reconstruction.

With $|{\mathbf{k}}_i|=k$, elastic scattering into a propagating wavevector ${\mathbf{k}}_s$ samples Fourier transfer $\mathbf{\kappa }={\mathbf{k}}_s-{\mathbf{k}}_i$. These transfers satisfy the displayed shifted sphere relation. This reciprocal-space construction appears in crystallographic diffraction and the Fourier diffraction theorem.

With the normal directed out of an obstacle $D$ into the fluid, a field solving $(\Delta +k^2)\psi =-Q$ outside $D$ has the representationHere $G_k$ is the outgoing Green function for the three-dimensional Helmholtz equation and the observation point lies outside the obstacle. The formula follows from Green second identity; the normal to the exterior fluid on its inner boundary is the negative of the stated obstacle normal. The surface term represents the scattered field.

For a scattered field satisfying ${\psi }_s=e^{ikr}{f}_{\infty }(\hat{\mathbf{r}})/r+O(r^{-2})$, the Kirchhoff–Helmholtz representation givesThe sign assumes the normal points out of the obstacle.

A Herglotz wave function is a superposition of incident plane waves,It solves the Helmholtz equation everywhere and is real analytic. The finite sphere area makes the density integrable, so each spatial derivative can be taken under the integral.

Pairing a far-field pattern with the conjugate density of a Herglotz wave function converts the angular integral into boundary data:This follows by substituting the surface representation of an obstacle far-field pattern and interchanging the integrals.

If measured far-field patterns determine a density $g$ and its Herglotz wave function equals a prescribed spherical wave on the obstacle boundary, that trace identity can locate the boundary. A star-shaped axisymmetric obstacle can be represented by one radial function $h(\theta )$. Numerically, expand $g$ and $h$ in suitable bases, match the far-field constraints, and minimize the complex boundary residual at angular collocation points, using regularization of an inverse problem for noise and small singular values. A single scalar pairing constraint alone does not determine an arbitrary density.

For a refractive index $n(\mathbf{r})$ and background wavenumber $k_0$, one sign convention defines the scattering potential by $V=k_0^2(1-n^2)$. The total field then satisfies $(\Delta +k_0^2)\psi =V\psi$.

The opposite convention $V=k_0^2(n^2-1)$ gives $(\Delta +k_0^2)\psi =-V\psi$ and reverses the sign attached to the corresponding Green-function integral. Either convention is valid when used consistently.

Using $V=k_0^2(n^2-1)$, the scalar Lippmann-Schwinger equation is $\psi ={\psi }_i+{\int }_D{G}_{k_0}V\psi$. The Born approximation replaces the total field inside the integral by the known incident field:For a unit plane wave, its far-field pattern is $(4\pi {)}^{-1}{\int }_{D}V({\mathbf{r}}^{\prime })e^{i{k}_0({\hat{\mathbf{r}}}_0-\hat{\mathbf{r}})\cdot {\mathbf{r}}^{\prime }}d{\mathbf{r}}^{\prime }$. Thus each incident direction samples a shifted sphere in the Fourier transform of the potential.

For real $h\in C_c^{\infty }(D)$, the displayed real compactly supported contrast has Fourier transform $\hat{V}(\mathbf{\kappa })=[|\mathbf{\kappa }{|}^4-4({\mathbf{k}}_i\cdot \mathbf{\kappa }{)}^2]\hat{h}(\mathbf{\kappa })$. This vanishes on the shifted Ewald sphere because $|\mathbf{\kappa }{|}^2+2{\mathbf{k}}_i\cdot \mathbf{\kappa }=0$ there. Thus nonzero contrasts can have zero fixed-frequency, fixed-incidence Born approximation data. A small amplitude preserves weak contrast and a positive refractive index.

The Born series is the Neumann series expansion of the Lippmann-Schwinger equation. Its terms describe successive orders of wave scattering; convergence is ensured, for example, by a scattering operator norm smaller than one.

The Rytov approximation writes a total wave field as $\psi ={\psi }_i{e}^{\varphi }$ and linearizes the equation for the complex logarithmic perturbation $\varphi$. At first order, ${\psi }_i\varphi$ equals the first Born approximation scattered field, soIt can remain useful when phase accumulation is appreciable but local scattering is weak.

If the first logarithmic perturbation ${\chi }_1$ is a centered linear functional of a Gaussian random field, the complex Gaussian exponential moment gives the displayed mean of the first Rytov approximation ${\psi }_R={\psi }_i{e}^{\mu {\chi }_1}$. This is the mean of the truncated logarithmic model. A nonzero mean of the omitted second logarithmic perturbation can change the true field mean at order ${\mu }^2$.

For a centered real scattering potential fluctuation $W$ and the imaginary part $b$ of the incident-field-weighted Green function, $\phi (\mathbf{r})=-\int b(\mathbf{r},{\mathbf{r}}^{\prime })W({\mathbf{r}}^{\prime })d{\mathbf{r}}^{\prime }$. Its mean vanishes, and its covariance function is the double integral of the two deterministic kernels against the covariance function of $W$. The correlation of the refractive index itself is insufficient when the scattering potential depends quadratically on that index.

The Rytov approximation omits the quadratic gradient source in the logarithmic wave perturbation equation. Its next correction solves ${\mathcal{L}}_i{\chi }_2=-\nabla {\chi }_1\cdot \nabla {\chi }_1$. Control of the propagated correction, weak amplitude fluctuations and avoidance of field zeros are needed. An appreciable smooth accumulated phase can be retained by exponentiation; small total phase is not the same condition as small local phase gradients.

The logarithmic wave perturbation measures relative complex amplitude and phase through $\psi ={\psi }_i{e}^{\chi }$. A continuous branch requires nonzero incident and total fields in the region considered. For the Helmholtz equation with scattering potential convention $L_0\psi =-V\psi$, its equation is $(\Delta +2\nabla \log {\psi }_i\cdot \nabla )\chi +\nabla \chi \cdot \nabla \chi =-V$. The dot product is bilinear, not a squared modulus.

Let ${\psi }_R={\psi }_i{e}^{{\varphi }_1}$ with unit-modulus incidence and ${\varphi }_1={\int }_{D}K(\mathbf{r},{\mathbf{r}}^{\prime })V({\mathbf{r}}^{\prime })d{\mathbf{r}}^{\prime }$. For a centered real Gaussian random field $V$, the exponential moment of a Gaussian linear functional givesHere $C_V$ is the autocorrelation function of a random field. Both the covariance $\mathbb{E}|{\varphi }_1{|}^2$ and the generally nonzero term $\mathrm{Re}\mathbb{E}{\varphi }_1^2$ enter the exponent; dropping the latter is not justified for an arbitrary finite real random medium.

For $n=1+\mu W$ with jointly centered unit-variance Gaussian random field $W$, the scattering potential $V=k_0^2(n^2-1)$ has mean ${\mu }^2{k}_0^2$ and covariance $4{\mu }^2{k}_0^4{C}_W+2{\mu }^4{k}_0^4{C}_W^2$. It is not Gaussian. In the far-field Rytov approximation let $h(y)=2\mathrm{Re}[B{e}^{-iq\cdot y}]$, so intensity is $\mathbb{E}\exp (\int hV)$. Expansion through order ${\mu }^2$ includes both the mean term and the covariance term shown. The remainder starts at order ${\mu }^4$ under suitable exponential integrability. Keeping a covariance correction while omitting the same-order mean term is inconsistent. The exact quadratic model uses the Gaussian quadratic exponential moment.

The far-field pattern is the directional coefficient $f_{\infty }(\hat{\mathbf{r}})$ in an outgoing asymptotic field $u_s(\mathbf{r})=e^{ikr}{f}_{\infty }(\hat{\mathbf{r}})/r+O(r^{-2})$.

For a bounded source region and fixed positive wavenumber, expand $|R\mathbf{\theta }-{\mathbf{r}}^{\prime }|=R-\mathbf{\theta }\cdot {\mathbf{r}}^{\prime }+O(R^{-1})$. In the outgoing Green function for the three-dimensional Helmholtz equation, this gives $G=e^{ikR}{e}^{-ik\mathbf{\theta }\cdot {\mathbf{r}}^{\prime }}/(4\pi R)+O(R^{-2})$. Volume integration gives the outgoing field's far-field pattern with an $O(R^{-2})$ remainder. The estimate is an asymptotic statement for fixed source size and frequency.

The Sommerfeld radiation condition selects outgoing solutions of the exterior Helmholtz equation. In three dimensions it requires $r({\partial }_{r}u-iku)\to 0$ as $r\to \infty$ uniformly in direction.

A problem is well posed in the sense of Hadamard when a solution exists for every admissible datum, is unique, and depends continuously on the data. Failure of any condition makes it ill posed.

Regularization replaces sensitive inversion by a controlled approximation. For a bounded linear operator between Hilbert spaces, a linear regularization is a family of bounded linear operators $R_{\alpha }$ satisfying $R_{\alpha }y\to A^{†}y$ as $\alpha \downarrow 0$ for each datum in the domain of the Moore–Penrose inverse of an operator. Noisy-data convergence additionally requires a suitable regularization parameter choice.^[ref] Nonlinear variational regularization can instead use nonquadratic penalties.

The noise-bias decomposition for linear regularization separates two costs: amplifying measurement noise and changing the exact-data reconstruction. Truncated singular value decomposition discards weak modes; Tikhonov regularization attenuates them smoothly. A quadratic norm penalty expresses a preference for smaller reconstructions. Neither method manufactures a missing null space component: with complete averaging, the minimum-norm least-squares solution for $(0.5,0.5{)}^T$ is $(0.5,0.5{)}^T$, even if the original image was $(1,0{)}^T$. The preference selects a representative; extra observations or justified constraints are needed to identify the original.

For Tikhonov regularization, the Tikhonov stability bound limits transmitted measurement noise of norm at most $\delta$ to $\delta /(2\sqrt{\alpha })$. Taking $\alpha \to 0$ while $\delta /\sqrt{\alpha }\to 0$ gives a sufficient rule for convergent regularization of an inverse problem. Holding $\alpha$ fixed leaves approximation bias; letting it vanish too quickly can admit overwhelming measurement noise.

For a positive self-adjoint linear operator $A$, Lavrentiev regularization shifts $A$ itself rather than $A^{\ast }A$. In a finite-dimensional positive-definite problem, writing $y^{\delta }=Ax+e$ gives error coefficients $(e_i-\alpha x_i)/({\lambda }_i+\alpha )$ in its orthonormal eigenbasis. Consequently $\Vert x_{\alpha }^{\delta }-x\Vert \le \delta /({\lambda }_1+\alpha )+\alpha \Vert x\Vert /({\lambda }_1+\alpha )\le \delta /\alpha +\alpha \Vert x\Vert /{\lambda }_1$. A known bound $X\ge \Vert x\Vert$ permits $\alpha =\sqrt{\delta {\lambda }_1/X}$, giving a robust bound $2\sqrt{\delta X/{\lambda }_1}$. For fixed positive ${\lambda }_1$, the sharper finite-dimensional estimate can give a linear noise rate with a smaller parameter; regularization is then a conditioning choice, not a cure for nonexistence of the inverse.

A linear regularization uses a family of bounded linear operators $R_{\alpha }$ such that $R_{\alpha }f\to K^{†}f$ for each $f\in \mathcal{D}(K^{†})$. A regularization parameter choice must balance approximation error with noise amplification to obtain a convergent regularization of an inverse problem.

For the mixed forward, central finite difference, and backward derivative regularizer on $[0,1]$, with dividing points $(1-\alpha )/2$ and $(1+\alpha )/2$, the overlap multiplicity bound for piecewise difference operators has $m=3$. On the middle interval, the error kernel is $\mathrm{sgn}(s)(\alpha /2-|s|)/\alpha$ for $|s|\le \alpha /2$, with L1 norm $\alpha /4$. Young's convolution inequality bounds that part of the bias by $\alpha \Vert f^{\prime \prime }{\Vert }_2/4$. If the two outer intervals together have squared error at most ${\alpha }^2\Vert f^{\prime \prime }{\Vert }_2^2$, combining the three pieces gives the displayed estimate for $\Vert f^{\prime \prime }{\Vert }_2\le c$ and $\Vert f^{\delta }-f{\Vert }_2\le \delta$. Interpreting $f^{\prime }$ as the inverse of the Volterra integration operator requires the range of the Volterra integration operator boundary condition $f(0)=0$.

Suppose a piecewise finite difference uses two translated evaluations divided by $\alpha$. If the pulled-back evaluation intervals cover almost every input point at most $m$ times, then $|a-b{|}^2\le 2(|a{|}^2+|b{|}^2)$ gives $\Vert R_{\alpha }f{\Vert }_2^2\le 2m\Vert f{\Vert }_2^2/{\alpha }^2$. This converts an interval-overlap count into an operator norm estimate without requiring pointwise values of an L2 space equivalence class outside the almost-everywhere evaluation rule.

A regularization parameter controls the tradeoff between approximation bias and data-noise amplification in an inverse problem. For early stopping of Landweber iteration with fixed step $\tau$, the convention $\alpha =1/(\tau n)$ associates a smaller parameter with more iterations.

A regularization $R_{\alpha }$ is convergent when its parameter can be chosen from the noise level and data so that the regularized reconstruction converges to the exact Moore–Penrose inverse of an operator solution as the noise level tends to zero.

For a bounded linear regularization operator $R_{\alpha }$ and noisy data with $\Vert f_{\delta }-f\Vert \le \delta$, the triangle inequality givesThe first term is noise amplification; the second is the approximation bias on exact data. Thus $\alpha (\delta )\to 0$ and $\delta \Vert R_{\alpha (\delta )}\Vert \to 0$ suffice for a convergent regularization of an inverse problem, provided $R_{\alpha }$ is consistent on exact data.

A spectral regularization method applies a bounded scalar filter to the spectrum of $A^{\ast }A$. In a singular system of a compact operator, it has the formwhere the filters approximate $1/\lambda$ while controlling amplification near $\lambda =0$.

Asymptotic regularization stops the gradient flow $x^{\prime }=-A^{\ast }(Ax-f)$ at $t=1/\alpha$, starting from zero. With the singular system of a compact operator convention $A{v}_j={\sigma }_j{u}_j$, its spectral filter givesThe scalar coefficient solves $c_j^{\prime }=-{\sigma }_j^2{c}_j+{\sigma }_j⟨f,u_j⟩$ with zero initial value. Since $1-e^{-s}\le \min (s,1)$ for $s\ge 0$, the operator norm is at most ${\alpha }^{-1/2}$. On the domain of the Moore–Penrose inverse of an operator, dominated convergence theorem of the squared spectral coefficients proves $R_{\alpha }f\to A^{†}f$. The noise-bias decomposition for linear regularization then gives noisy-data convergence when $\delta /\sqrt{\alpha }\to 0$.

For a singular value decomposition $A{v}_j={\sigma }_j{u}_j$, truncated SVD with cutoff $\tau >0$ estimates the inverse action by $x_{\tau }={\sum }_{{\sigma }_j\ge \tau }{\sigma }_j^{-1}⟨b,u_j⟩v_j$. Discarding small singular values limits amplification of data errors; this is a spectral regularization method.

Truncated singular value decomposition retains only singular components above a threshold, or equivalently only a finite leading set of singular vectors.

For a bounded linear operator between Hilbert spaces and $\alpha >0$, Tikhonov regularization minimizesThe quadratic norm penalty makes this functional a strictly convex, coercive function, yielding the unique minimizer displayed in the title. Here $A^{\ast }$ is the adjoint operator and $I$ the identity operator. Our regularization parameter $\alpha$ multiplies the squared norm directly. Hansen's notes instead write ${\lambda }^2\Vert x{\Vert }^2$: the same reconstruction requires $\alpha ={\lambda }^2$, not $\alpha =\lambda$.^[ref]

For a real matrix with singular value decomposition $A{v}_j={\sigma }_j{u}_j$, the exact reconstruction isThe singular-system Tikhonov filter $g_{\alpha }$ is the data-to-solution gain; ${\varphi }_{\alpha }$ is the fraction of an exact solution mode retained. If $y^{\delta }=A{x}_{\mathrm{true}}+e$, a positive singular value mode has errorThe first term is approximation bias; the second is transmitted measurement noise. Suppressing a weak mode suppresses its true signal too.

For the two-pixel matrix in the inverse problem example and $\alpha =0.01$ ($\lambda =0.1$), the mean and difference attenuation factors are $100/101$ and $1/2$. Since $y^{\delta }=q_{+}/\sqrt{2}+0.12q_{-}/\sqrt{2}$, the reconstruction isIts noise contribution is $(0.05,-0.05{)}^T$, half the direct inverse's, but its exact-data reconstruction is $(0.74505,0.24505{)}^T$. Consequently its total error Euclidean norm, about $0.28293$, exceeds the direct inverse's $0.14142$: this parameter sacrifices too much contrast. Taking $\alpha =0.001$ instead gives $(1.04496,-0.04595{)}^T$ and error about $0.06429$. Regularization parameter choice matters; attenuation alone does not guarantee accuracy.