= Tikhonov regularization
{c}
{title2=$x_\alpha^\delta=(A^*A+\alpha I)^{-1}A^*y^\delta$}
For a <bounded linear operator> between <Hilbert spaces> and $\alpha>0$, Tikhonov regularization minimizes
$$
\lVert Ax-y^\delta\rVert^2+\alpha\lVert x\rVert^2.
$$
The <quadratic norm penalty> makes this functional a <strictly convex>, <coercive function>, yielding the unique minimizer displayed in the title. Here $A^*$ is the <adjoint operator> and $I$ the <identity operator>. Our <regularization parameter> $\alpha$ multiplies the squared <norm> directly. Hansen's notes instead write $\lambda^2\|x\|^2$: the same reconstruction requires $\alpha=\lambda^2$, not $\alpha=\lambda$.https://www.imm.dtu.dk/~pcha/Regutools/RTv4manual.pdf[Per Christian Hansen, Regularization Tools, §§2.4 and 2.7.1]{ref}
For a real <matrix> with <singular value decomposition> $Av_j=\sigma_j u_j$, the exact reconstruction is
$$
x_\alpha^\delta=\sum_{\sigma_j>0}\underbrace{\frac{\sigma_j}{\sigma_j^2+\alpha}}_{g_\alpha(\sigma_j)}(u_j^Ty^\delta)v_j,\qquad
\phi_\alpha(\sigma)=\sigma g_\alpha(\sigma)=\frac{\sigma^2}{\sigma^2+\alpha}.
$$
The <singular-system Tikhonov filter> $g_\alpha$ is the data-to-solution gain; $\phi_\alpha$ is the fraction of an exact solution mode retained. If $y^\delta=Ax_{\mathrm{true}}+e$, a positive <singular value> mode has error
$$
v_j^T(x_\alpha^\delta-x_{\mathrm{true}})
=-\frac{\alpha}{\sigma_j^2+\alpha}v_j^Tx_{\mathrm{true}}
+\frac{\sigma_j}{\sigma_j^2+\alpha}u_j^Te.
$$
The first term is approximation bias; the second is transmitted <measurement noise>. Suppressing a weak mode suppresses its true signal too.
For the two-pixel <matrix> in the <inverse problem> example and $\alpha=0.01$ ($\lambda=0.1$), the mean and difference attenuation factors are $100/101$ and $1/2$. Since $y^\delta=q_+/\sqrt2+0.12q_-/\sqrt2$, the reconstruction is
$$
x_{0.01}^\delta=\frac{100}{101}\frac{q_+}{\sqrt2}+\frac35\frac{q_-}{\sqrt2}
=\begin{pmatrix}803/1010\\197/1010\end{pmatrix}
\approx\begin{pmatrix}0.79505\\0.19505\end{pmatrix}.
$$
Its noise contribution is $(0.05,-0.05)^T$, half the direct inverse's, but its exact-data reconstruction is $(0.74505,0.24505)^T$. Consequently its total error <Euclidean norm>, about $0.28293$, exceeds the direct inverse's $0.14142$: this parameter sacrifices too much contrast. Taking $\alpha=0.001$ instead gives $(1.04496,-0.04595)^T$ and error about $0.06429$. <Regularization parameter choice> matters; attenuation alone does not guarantee accuracy.
\Image[/inverse-problem-blur-and-regularization.png]
{title=Tikhonov attenuation reduces difference-mode noise while biasing the reconstructed contrast}
Back to article page
