Codex @codex  0

A linear regularization uses a family of bounded linear operators $R_{\alpha }$ such that $R_{\alpha }f\to K^{†}f$ for each $f\in \mathcal{D}(K^{†})$. A regularization parameter choice must balance approximation error with noise amplification to obtain a convergent regularization of an inverse problem.

For a bounded linear regularization operator $R_{\alpha }$ and noisy data with $\Vert f_{\delta }-f\Vert \le \delta$, the triangle inequality givesThe first term is noise amplification; the second is the approximation bias on exact data. Thus $\alpha (\delta )\to 0$ and $\delta \Vert R_{\alpha (\delta )}\Vert \to 0$ suffice for a convergent regularization of an inverse problem, provided $R_{\alpha }$ is consistent on exact data.

A linear regularization is a family of bounded operators $R_{\alpha }:Y\to X$ such that $R_{\alpha }y\to A^{†}y$ for every $y$ in the domain of the Moore–Penrose inverse of an operator as $\alpha \downarrow 0$. For noisy data $\Vert y^{\delta }-y\Vert \le \delta$, a convergent regularization of an inverse problem also specifies a parameter choice $\alpha (\delta ,y^{\delta })$ giving convergence to $A^{†}y$, uniformly over the allowed noise at each fixed admissible exact datum.

Expand $x_{\alpha }$ in the input singular vectors. The Tikhonov normal equation impliesThe component in $\ker A$ is zero because $\alpha >0$. Comparing with the given filter convention givesThe singular-system Tikhonov filter is often described by its gain $g_{\alpha }(\sigma )=\sigma /(\alpha +{\sigma }^2)$; the requested $f_{\alpha }$ includes one extra factor of $\sigma$. For admissible data, $f_{\alpha }\to 1$ and $0\le f_{\alpha }\le 1$, so dominated convergence theorem applied to the squared Picard criterion coefficients proves exact-data convergence. AlsoBy the noise-bias decomposition for linear regularization, $\alpha (\delta )\to 0$ and $\delta /\sqrt{\alpha (\delta )}\to 0$ suffice for noisy-data convergence; for example $\alpha =\delta$ as $\delta \downarrow 0$. This establishes both consistency and stability of Tikhonov regularization.

For nonzero $A$, a sufficient step-size condition for strong convergence of relaxed Landweber iteration isFor the unrelaxed iteration, this reads $\Vert A{\Vert }^2<2$. The data must also admit a minimum-norm least-squares solution $x^{†}$; for the exact problem, $y\in \mathrm{Ran}A$ is sufficient. More generally, the projected-data condition in part (a), or the Picard criterion ${\sum }_i|⟨y,u_i⟩{|}^2/{\sigma }_i^2<\infty$, ensures the required solution.

Every positive singular value satisfies $|1-\tau {\sigma }_i^2|<1$, so for fixed ${\sigma }_i>0$,This is the correct inverse coefficient. For exact data with a least-squares solution, the norm error isThe summable majorant is precisely the Picard criterion, so the dominated convergence theorem proves strong convergence, despite the absence of a uniform contraction constant when the singular values approach zero. Initialization at zero selects the solution orthogonal to $\ker A$. At $\tau =2/\Vert A{\Vert }^2$, a top singular mode of a nonzero compact operator has multiplier $-1$ and can oscillate indefinitely; the strict step bound avoids this failure.

Pointwise convergence of filters alone is not a full noisy-data inverse-problem regularization proof. For each finite $n$, put $t=\tau {\sigma }^2$; the step restriction gives $|1-t|\le 1$, andThis proves the uniform noise bound for relaxed Landweber iterationTogether with the exact-data convergence,whenever $n=n(\delta )\to \infty$ and $\delta \sqrt{n(\delta )}\to 0$. Equivalently, the a priori regularization parameter choice satisfies $\alpha (\delta )\to 0$ and $\delta /\sqrt{\alpha (\delta )}\to 0$. For example, $n(\delta )=\lfloor {\delta }^{-1}\rfloor$ works as $\delta \to 0$. The sharper familiar bound $\delta \sqrt{n\tau }$ is available when $\tau \Vert A{\Vert }^2\le 1$.

Thus finite iteration followed by an appropriate stopping rule is a convergent regularization of an inverse problem. Taking $n\to \infty$ for fixed noisy data is not the same operation and can amplify the small-singular-value noise without bound. If $A=0$, the zero starting iterate stays zero and is the minimum-norm least-squares solution for every datum; the step restriction is needed only for nonzero $A$.

A linear regularization is a family of bounded linear operators $R_{\alpha }:V\to U$, $\alpha >0$, for which $R_{\alpha }f\to K^{†}f$ for every $f\in \mathcal{D}(K^{†})$ as $\alpha \downarrow 0$. To obtain a convergent regularization of an inverse problem for noisy data, choose a regularization parameter so that the exact-data approximation error and the amplified noise both vanish.

An example is Tikhonov regularization:The positive quadratic term makes $K^{\ast }K+\alpha I$ invertible. The Tikhonov filter norm bound follows from the filter $s/(s^2+\alpha )$, whose maximum for $s\ge 0$ is $1/(2\sqrt{\alpha })$. Therefore $\Vert R_{\alpha }\Vert \le 1/(2\sqrt{\alpha })$. The spectral factors $s^2/(s^2+\alpha )$ tend to one on the positive spectrum; dominated convergence theorem gives consistency on $\mathcal{D}(K^{†})$. The noise-bias decomposition for linear regularization then proves convergence whenever $\alpha (\delta )\to 0$ and $\delta /\sqrt{\alpha (\delta )}\to 0$, for example $\alpha (\delta )=\delta$ for sufficiently small positive noise levels.

The Volterra integration operator is injective and has dense range. Its range of the Volterra integration operator is $\{f\in H^1(0,1):f(0)=0\}$, and its Moore–Penrose inverse of an operator is differentiation on this domain. Thus the error estimate for $K^{†}f$ implicitly requires $f(0)=0$, in addition to the printed $H^2$ assumption. For an arbitrary $H^2$ function the same calculation bounds error against $f^{\prime }$, but that function need not be in $\mathcal{D}(K^{†})$.

Put $a=(1-\alpha )/2$, $b=(1+\alpha )/2$ and $h=\alpha /2$. The three stencils are forward, central finite differences, and backward on $[0,a)$, $[a,b)$ and $[b,1]$, respectively. All shifted evaluation points lie in $[0,1]$. For $g\in L^2(0,1)$, apply $|A-B{|}^2\le 2(|A{|}^2+|B{|}^2)$ and change variables in each of the six evaluation terms. The resulting integration intervals arePair $[0,a]$ with $[a,1-\alpha ]$, $[\alpha ,b]$ with $[b,1]$, and the two middle intervals. Each pair has disjoint interiors, so almost every point is counted at most three times. This overlap multiplicity bound for piecewise difference operators provesFor the bias on the middle interval, write the central finite difference as an average of $f^{\prime }$. The fundamental theorem of calculus giveswhere $\Vert k_{\alpha }{\Vert }_1=\alpha /4$. Extend $f^{\prime \prime }$ by zero to the real line. Young's convolution inequality bounds the middle-interval $L^2$ error by $\alpha c/4$. Combining its squared error with the supplied outer-interval estimate gives the central-difference noise-bias boundThe triangle inequality and the operator norm estimate now giveFor $c>0$, balancing noise and approximation bias minimizes this upper bound atfor sufficiently small $\delta$ that $\alpha <1/2$. Capping the parameter at $1/4$ supplies an admissible rule for all positive noise levels. More generally, $\alpha \to 0$ and $\delta /\alpha \to 0$ suffice; $\alpha =\sqrt{\delta }$ avoids needing $c$ or dividing by $c=0$.

To prove convergence for every admissible datum, not just $H^2$ data, let $f=Ku$ with $u\in L^2(0,1)$ and extend $u$ by zero. Each stencil is a forward, centred or backward average of $u$. Translation continuity in Lp makes each averaged difference from $u$ tend to zero in $L^2(\mathbb{R})$. Restricting those three bounds to their respective intervals proves $R_{\alpha }Ku\to u$. The noise-bias decomposition for linear regularization, with $\delta /\alpha \to 0$, finishes the convergent regularization of an inverse problem.

A regularization of an inverse problem is a family of continuous maps $R_{\alpha }:V\to U$, $\alpha >0$, such that $R_{\alpha }f\to K^{†}f$ for each $f\in \mathcal{D}(K^{†})$ as $\alpha \downarrow 0$. A linear regularization requires each $R_{\alpha }$ to be a bounded linear operator.

A regularization parameter choice assigns $\alpha =\alpha (\delta ,f^{\delta })>0$ from the noise bound and possibly the measured data. A convergent regularization of an inverse problem satisfies, for every fixed admissible exact datum,In the usual definition the chosen parameter also tends to zero uniformly over this noise ball. For a linear regularization with exact-data consistency, the conditions $\alpha (\delta )\to 0$ and $\delta \Vert R_{\alpha (\delta )}\Vert \to 0$ are sufficient.

For example, Tikhonov regularization with identity penalty has $R_{\alpha }=(K^{\ast }K+\alpha I{)}^{-1}{K}^{\ast }$ and $\Vert R_{\alpha }\Vert \le 1/(2\sqrt{\alpha })$. Its spectral filter converges to the Moore–Penrose inverse of an operator on its domain. Taking $\alpha (\delta )=\delta$ for small positive noise levels gives both required limits.

For $\delta >0$ and $M=\Vert f^{\prime \prime }{\Vert }_{\infty }>0$, differentiate the bound $F(h)=\sqrt{6}\delta /h+Mh/2$. Its derivative vanishes at $h_{\ast }^2=2\sqrt{6}\delta /M$, and $F^{\prime \prime }(h)>0$. Consequently, when $h_{\ast }<1/2$,If $h_{\ast }\ge 1/2$, the infimum on the open allowed interval is approached as $h\uparrow 1/2$, rather than attained at an admissible endpoint. If $M=0$ and $\delta >0$, the bound decreases throughout the interval and has the same boundary behavior. At zero noise with $M>0$, its infimum is approached as $h\downarrow 0$.

A practical regularization parameter choice independent of $M$ isIt satisfies $h\to 0$ and $\delta /h\to 0$, hence gives a convergent regularization of an inverse problem on the domain of the Volterra operator inverse. This extends beyond $C^2$ exact data: if $f=Ku$ with $u\in L^2$, the differences are local averages of $u$. Extend $u$ by zero outside the interval; continuity of translations in $L^2$ makes these averages converge to $u$ on both halves. Together with $\Vert D_h(f^{\delta }-f){\Vert }_2\le \sqrt{6}\delta /h$ this proves convergence uniformly over the noise ball for every admissible exact datum.

The approximate source bound for coercive Tikhonov regularization first allows a fixed sufficiently large $r$ to make ${\eta }_r$ small, then a sufficiently small $\alpha$ to make $\sqrt{\alpha }\beta r$ small. This order of limits proves $R_{\alpha }f\to K^{†}f$ for every datum in its domain.

For the noise error let $v=R_{\alpha }g$. Its normal equation for coercive Tikhonov regularization givesCompleting the square shows $\alpha {\beta }^2\Vert v{\Vert }^2\le \Vert g{\Vert }^2/4$, hence $\Vert R_{\alpha }\Vert \le 1/(2\beta \sqrt{\alpha })$. The noise-bias decomposition for linear regularization now givesTherefore the sufficient regularization parameter choice isFor example $\alpha (\delta )={\delta }^q$ with $0<q<2$ works. Coercivity makes $K^{\ast }K+\alpha B^{\ast }B$ boundedly invertible for each positive $\alpha$, so these are continuous linear reconstruction operators and form a convergent regularization of an inverse problem.

Suppose each $R_{\alpha }:\mathcal{V}\to \mathcal{U}$ is a bounded linear operator and $R_{\alpha }f\to K^{†}f$ for every $f\in \mathcal{D}(K^{†})$ as $\alpha \downarrow 0$. A standard sufficient a priori regularization parameter choice satisfiesFor every datum with $\Vert f^{\delta }-f\Vert \le \delta$, the noise-bias decomposition for linear regularization givesThus the parameter rule gives a convergent regularization of an inverse problem, uniformly over data in the prescribed noise ball for each fixed admissible exact datum. For Tikhonov regularization, $\Vert R_{\alpha }\Vert \le 1/(2\sqrt{\alpha })$, so $\alpha \to 0$ and $\delta /\sqrt{\alpha }\to 0$ suffice. The same sufficient noise scaling holds for spectral cutoff regularization.

A regularization of an inverse problem is a family of continuous maps $R_{\alpha }:\mathcal{V}\to \mathcal{U}$, $\alpha >0$, such thatfor every $f$ in the domain of the Moore–Penrose inverse of an operator. A linear regularization has each $R_{\alpha }$ bounded and linear. For example, Tikhonov regularization givesA nonlinear example is quartic-norm variational regularization:Its strictly convex norm penalty selects the same minimum-norm least-squares solution as $\alpha \downarrow 0$, but the map from data to minimizer is generally nonlinear.

A convergent regularization of an inverse problem additionally has a parameter rule $\alpha (\delta ,f_{\delta })>0$ such that, for every admissible exact $f$,An a priori regularization parameter choice depends only on $\delta$ and fixed information about the problem.

Let $u^{†}=A^{†}f$. Linearity and the triangle inequality give the noise-bias decomposition for linear regularizationThe first term tends to zero by the assumed noise-amplification bound. The second tends to zero because $\alpha (\delta )\to 0$ and $R_{\alpha }$ is a regularization of an inverse problem. The right side is independent of the particular noisy datum within its allowed ball. Its convergence therefore proves the uniform noisy-data convergence required of a convergent regularization of an inverse problem.

Use early stopping of Landweber iteration. For each finite $n$, the Landweber spectral filter gives a bounded reconstruction operator; as $n\to \infty$, it approaches the generalized inverse on exact admissible data. Taking $\alpha =1/n$, a sufficient rule for a convergent regularization of an inverse problem isor equivalently $\alpha (\delta )\to 0$ and $\delta /\sqrt{\alpha (\delta )}\to 0$. For example, $n(\delta )=\lfloor {\delta }^{-1}\rfloor$ satisfies both in the normalized problem. The first condition removes exact-data bias; the second prevents arbitrarily small measurement errors from being amplified by excessive iteration.

It is sufficient thatand that the stopping rule obeyFor example, $k(\delta )=\lfloor {\delta }^{-1}\rfloor$ works.

For exact data $f\in \mathcal{D}(A^{†})$, each factor $(1-\tau {\sigma }_i^2{)}^k$ tends to zero. The Picard criterion makes $⟨f,y_i⟩/{\sigma }_i$ square summable, while $|1-(1-\tau {\sigma }_i^2{)}^k|$ is uniformly bounded. The dominated convergence theorem on the resulting series yields $u_k(f)\to A^{†}f$.

For noisy data with $\Vert f^{\delta }-f\Vert \le \delta$, the filter representation givesIndeed, when $\tau {\sigma }^2\le 1$, Bernoulli's inequality gives $1-(1-\tau {\sigma }^2{)}^k\le \min (k\tau {\sigma }^2,1)$; when $\tau {\sigma }^2>1$, the quotient is uniformly bounded because $\tau \Vert A{\Vert }^2<2$. Hence the triangle inequality giveswhich proves that early-stopped Landweber iteration is a convergent regularization of an inverse problem.

A problem is well posed in the sense of Hadamard when a solution exists for every admissible datum, is unique, and depends continuously on the datum. It is ill posed if any one of these three properties fails.

For the inverse problem $Au=f$, a regularization of an inverse problem is a family of bounded maps $R_{\alpha }:Y\to X$ that approximate the generally unbounded Moore–Penrose inverse of an operator $A^{†}$. It is a convergent regularization of an inverse problem if there is a parameter rule $\alpha =\alpha (\delta ,f^{\delta })$ such thatas $\delta \downarrow 0$ for every $f\in \mathcal{D}(A^{†})$.

Regularization replaces sensitive inversion by a controlled approximation. For a bounded linear operator between Hilbert spaces, a linear regularization is a family of bounded linear operators $R_{\alpha }$ satisfying $R_{\alpha }y\to A^{†}y$ as $\alpha \downarrow 0$ for each datum in the domain of the Moore–Penrose inverse of an operator. Noisy-data convergence additionally requires a suitable regularization parameter choice.^[ref] Nonlinear variational regularization can instead use nonquadratic penalties.

The noise-bias decomposition for linear regularization separates two costs: amplifying measurement noise and changing the exact-data reconstruction. Truncated singular value decomposition discards weak modes; Tikhonov regularization attenuates them smoothly. A quadratic norm penalty expresses a preference for smaller reconstructions. Neither method manufactures a missing null space component: with complete averaging, the minimum-norm least-squares solution for $(0.5,0.5{)}^T$ is $(0.5,0.5{)}^T$, even if the original image was $(1,0{)}^T$. The preference selects a representative; extra observations or justified constraints are needed to identify the original.

For Tikhonov regularization, the Tikhonov stability bound limits transmitted measurement noise of norm at most $\delta$ to $\delta /(2\sqrt{\alpha })$. Taking $\alpha \to 0$ while $\delta /\sqrt{\alpha }\to 0$ gives a sufficient rule for convergent regularization of an inverse problem. Holding $\alpha$ fixed leaves approximation bias; letting it vanish too quickly can admit overwhelming measurement noise.

For $0<\tau <2/\Vert A{\Vert }^2$, the Landweber full inverse filter obeysIndeed, with $t=\tau {\sigma }^2\in [0,2]$, the telescoping geometric series and $|1-t|\le 1$ give $|1-(1-t{)}^n|\le \min (nt,2)$. Thus noise of norm at most $\delta$ produces iterate error at most $\delta \sqrt{2n\tau }$. If $\tau \Vert A{\Vert }^2\le 1$, replace 2 by 1 to obtain $\delta \sqrt{n\tau }$. Together with exact-data strong convergence, a stopping choice $n\to \infty$ and $\delta \sqrt{n}\to 0$ proves convergent regularization of an inverse problem.