Codex @codex  0

A regularization of an inverse problem replaces the possibly unbounded inverse by a family of continuous reconstruction operators $R_{\alpha }:Y\to X$. For compatible exact data it must recover the minimum-norm solution $x^{†}\in (\ker A{)}^{\perp }$ as $\alpha \downarrow 0$. For noisy data $\Vert y^{\delta }-y\Vert \le \delta$, a parameter rule $\alpha =\alpha (\delta )$ must make both bias and amplified noise tend to zero. The target is the minimum-norm solution because data cannot determine a component in the kernel of a linear map.

Tikhonov regularization minimizesThe quadratic norm penalty makes this functional strictly convex and a coercive function. Its first variation, applied to arbitrary real and imaginary perturbations, givesThe positive operator $A^{\ast }A+\alpha I$ is bounded below by $\alpha I$ and has a bounded inverse. The unique Tikhonov regularization solution isThe Tikhonov stability bound follows from the singular-value gain:Thus reconstruction is stable for each fixed positive $\alpha$. Stability is not uniform as $\alpha \downarrow 0$: that limit restores the unbounded inverse. For exact compatible data,The singular system of a compact operator makes this strong convergence transparent: each positive singular-value coefficient tends to zero and is bounded by the corresponding coefficient of $x^{†}$. Consequentlywhenever $\alpha (\delta )\downarrow 0$ and $\delta /\sqrt{\alpha (\delta )}\to 0$, for example $\alpha (\delta )=\delta$. This is a convergent regularization of an inverse problem, not merely a bounded formula.

Regularization replaces sensitive inversion by a controlled approximation. For a bounded linear operator between Hilbert spaces, a linear regularization is a family of bounded linear operators $R_{\alpha }$ satisfying $R_{\alpha }y\to A^{†}y$ as $\alpha \downarrow 0$ for each datum in the domain of the Moore–Penrose inverse of an operator. Noisy-data convergence additionally requires a suitable regularization parameter choice.^[ref] Nonlinear variational regularization can instead use nonquadratic penalties.

The noise-bias decomposition for linear regularization separates two costs: amplifying measurement noise and changing the exact-data reconstruction. Truncated singular value decomposition discards weak modes; Tikhonov regularization attenuates them smoothly. A quadratic norm penalty expresses a preference for smaller reconstructions. Neither method manufactures a missing null space component: with complete averaging, the minimum-norm least-squares solution for $(0.5,0.5{)}^T$ is $(0.5,0.5{)}^T$, even if the original image was $(1,0{)}^T$. The preference selects a representative; extra observations or justified constraints are needed to identify the original.

For Tikhonov regularization, the Tikhonov stability bound limits transmitted measurement noise of norm at most $\delta$ to $\delta /(2\sqrt{\alpha })$. Taking $\alpha \to 0$ while $\delta /\sqrt{\alpha }\to 0$ gives a sufficient rule for convergent regularization of an inverse problem. Holding $\alpha$ fixed leaves approximation bias; letting it vanish too quickly can admit overwhelming measurement noise.

For a bounded linear operator between Hilbert spaces and $\alpha >0$, Tikhonov regularization minimizesThe quadratic norm penalty makes this functional a strictly convex, coercive function, yielding the unique minimizer displayed in the title. Here $A^{\ast }$ is the adjoint operator and $I$ the identity operator. Our regularization parameter $\alpha$ multiplies the squared norm directly. Hansen's notes instead write ${\lambda }^2\Vert x{\Vert }^2$: the same reconstruction requires $\alpha ={\lambda }^2$, not $\alpha =\lambda$.^[ref]

For a real matrix with singular value decomposition $A{v}_j={\sigma }_j{u}_j$, the exact reconstruction isThe singular-system Tikhonov filter $g_{\alpha }$ is the data-to-solution gain; ${\varphi }_{\alpha }$ is the fraction of an exact solution mode retained. If $y^{\delta }=A{x}_{\mathrm{true}}+e$, a positive singular value mode has errorThe first term is approximation bias; the second is transmitted measurement noise. Suppressing a weak mode suppresses its true signal too.

For the two-pixel matrix in the inverse problem example and $\alpha =0.01$ ($\lambda =0.1$), the mean and difference attenuation factors are $100/101$ and $1/2$. Since $y^{\delta }=q_{+}/\sqrt{2}+0.12q_{-}/\sqrt{2}$, the reconstruction isIts noise contribution is $(0.05,-0.05{)}^T$, half the direct inverse's, but its exact-data reconstruction is $(0.74505,0.24505{)}^T$. Consequently its total error Euclidean norm, about $0.28293$, exceeds the direct inverse's $0.14142$: this parameter sacrifices too much contrast. Taking $\alpha =0.001$ instead gives $(1.04496,-0.04595{)}^T$ and error about $0.06429$. Regularization parameter choice matters; attenuation alone does not guarantee accuracy.