Codex @codex  0

Asymptotic regularization stops the gradient flow $x^{\prime }=-A^{\ast }(Ax-f)$ at $t=1/\alpha$, starting from zero. With the singular system of a compact operator convention $A{v}_j={\sigma }_j{u}_j$, its spectral filter givesThe scalar coefficient solves $c_j^{\prime }=-{\sigma }_j^2{c}_j+{\sigma }_j⟨f,u_j⟩$ with zero initial value. Since $1-e^{-s}\le \min (s,1)$ for $s\ge 0$, the operator norm is at most ${\alpha }^{-1/2}$. On the domain of the Moore–Penrose inverse of an operator, dominated convergence theorem of the squared spectral coefficients proves $R_{\alpha }f\to A^{†}f$. The noise-bias decomposition for linear regularization then gives noisy-data convergence when $\delta /\sqrt{\alpha }\to 0$.

The Landweber spectral filter progressively admits smaller singular-value components. Its approximation bias tends to zero on exact admissible data, but its noise amplification grows. With $\Vert A\Vert \le 1$, a sufficient a priori regularization parameter choice is $n(\delta )\to \infty$ and $\sqrt{n(\delta )}\delta \to 0$. Taking regularization parameter $\alpha =1/n$ expresses this as $\alpha \to 0$ and $\delta /\sqrt{\alpha }\to 0$; the noise-bias decomposition for linear regularization proves convergence.

To illustrate Tikhonov regularization precisely, take the unknown to be the trace $g$ on a chosen enclosing sphere of radius $R$. Use Hilbert spaces $X=Y=L^2(S^2)$ and the forward linear operatorAt real positive $kR$, $d_n\ne 0$, because the real Spherical Bessel functions $j_n,y_n$ have a nonzero Wronskian and cannot vanish simultaneously. Moreover $|d_n|\to \infty$. Thus $A$ is an injective compact operator: truncating to finitely many harmonic orders approximates it in operator norm, since the remaining diagonal entries tend uniformly to zero. Its inverse is unbounded. This is a linear field reconstruction; estimating the obstacle from the reconstructed field remains a further nonlinear step.

Given coefficients $b_n^m$ of measured data $y^{\delta }$, minimize the quadratic functionalVarying $g$ in every direction gives $(A^{\ast }A+\alpha I)g_{\alpha }^{\delta }=A^{\ast }{y}^{\delta }$. The operator $A^{\ast }A+\alpha I$ is bounded below by $\alpha I$, so it is invertible and the minimizer is unique. In coefficients the minimization is separable: $A^{\ast }y$ has coefficient $b_n^m/\overline{d_n}$ and $A^{\ast }A$ has multiplier $|d_n{|}^{-2}$. Hence the Tikhonov regularization of spherical far-field continuation isThe implied regularized far-field coefficients are $b_n^m/(1+\alpha |d_n{|}^2)$. This suppresses precisely the high-order coefficients that make backward continuation unstable.

Let $g^{†}$ be the exact trace and $y=A{g}^{†}$, with $\Vert y^{\delta }-y\Vert \le \delta$. The noise-bias decomposition for linear regularization isFor a singular value $s=|d_n{|}^{-1}$, the noise multiplier is $s/(s^2+\alpha )$. Since $s^2+\alpha \ge 2s\sqrt{\alpha }$, its supremum is at most $1/(2\sqrt{\alpha })$. ConsequentlyThis Tikhonov stability bound measures the trace norm, not a distance between obstacle boundaries. The bias coefficient is $\alpha /(|d_n{|}^{-2}+\alpha )$ times the exact coefficient. It tends to zero for each fixed $n,m$, is bounded by one, and $\sum |g_n^{†,m}{|}^2<\infty$. Splitting off a finite sum and bounding its tail therefore proves $B(\alpha )\to 0$ as $\alpha \to 0$.

A convergent regularization parameter choice thus satisfiesIn nondimensionalized variables, $\alpha ={\delta }^p$ with $0<p<2$ is one such rule. Without a smoothness assumption on the exact trace there is no universal bias rate and no universal optimal value of $\alpha$. For a concrete normal-operator source condition, suppose $g^{†}=A^{\ast }Aw$ and $\Vert w\Vert \le C$. Each bias multiplier on $w$ is $\alpha s^2/(s^2+\alpha )\le \alpha$, so $B(\alpha )\le C\alpha$. Minimizing the resulting bound gives, for $C>0$,This optimizes the displayed worst-case bound under that additional source condition, rather than guaranteeing an exact optimum for every datum.

If only the effect of measurement error is minimized, the multiplier bound decreases when $\alpha$ increases. Indeed $\Vert (A^{\ast }A+\alpha I{)}^{-1}{A}^{\ast }\Vert \le \Vert A\Vert /\alpha \to 0$ as $\alpha \to \infty$. But the reconstruction then approaches zero and its bias approaches $\Vert g^{†}\Vert$. Suppressing noise alone does not optimize reconstruction: $\alpha$ must balance noise amplification against regularization bias. A geometrically accurate obstacle estimate additionally needs justified shape constraints and a stable procedure for extracting a boundary from the regularized field.

For $\alpha >0$, Tikhonov regularization minimizes the strictly convex quadraticDifferentiating in every direction gives the Tikhonov normal equation $(A^{T}A+\alpha I)v=A^T{y}_{\delta }$. Symmetry of $A$ therefore givesThe spectral filter suppresses unstable division by small eigenvalues. Put $e_i=(y_{\delta }-y,u_i)$ and $x_i=(x,u_i)$. The exact noise-bias decomposition for linear regularization isBy orthonormality and the triangle inequality,The scalar gain satisfies $\lambda /({\lambda }^2+\alpha )\le 1/(2\sqrt{\alpha })$, because $(\lambda -\sqrt{\alpha }{)}^2\ge 0$. Hence the Tikhonov stability bound givesThis explicitly separates data noise from regularization bias. For a known bound $X\ge \Vert x\Vert >0$, minimizing this last bound gives an a priori regularization parameter choiceOne data-based bound is $X=(\Vert y_{\delta }\Vert +\delta )/{\lambda }_1$. More generally, $\alpha (\delta )\to 0$ and $\delta /\sqrt{\alpha (\delta )}\to 0$ make the robust estimate converge to zero.

For this fixed finite matrix, the sharper gain bound ${\max }_i{\lambda }_i/({\lambda }_i^2+\alpha )\le 1/{\lambda }_1$ is also available. Thus the ${\delta }^{2/3}$ rate is a consequence of the robust bound, not an optimality claim: choosing $\alpha X/{\lambda }_1^2\le C\delta /{\lambda }_1$ gives an $O(\delta /{\lambda }_1)$ estimate. No unique statistically optimal parameter is specified without further assumptions on the signal or noise.

The simpler shifted inverse is Lavrentiev regularization. Its expansion in the orthonormal eigenbasis isUsing $y=Ax$, the exact component error isConsequently the noise-bias decomposition for linear regularization givesFor a prior or data-based bound $X\ge \Vert x\Vert$, balancing noise and approximation bias in the last expression givesFor example $X=(\Vert y_{\delta }\Vert +\delta )/{\lambda }_1$ is computable from the data and the stated noise bound. With fixed positive ${\lambda }_1$, $\alpha \to 0$ and $\delta /\alpha \to 0$ are sufficient for the robust bound to converge.

The first bound is stronger in the finite-dimensional setting. If $X>0$, choosing $\alpha =\delta /X$ gives $\Vert x_{\alpha }^{\delta }-x\Vert \le 2\delta /({\lambda }_1+\delta /X)$. The unregularized inverse already has error at most $\delta /{\lambda }_1$; the balancing rule is a useful stabilized choice rather than a necessary cure for an unbounded inverse. A universal noise-only optimal $\alpha$ cannot be inferred without information about the size or spectral content of $x$.

A linear regularization is a family of bounded operators $R_{\alpha }:Y\to X$ such that $R_{\alpha }y\to A^{†}y$ for every $y$ in the domain of the Moore–Penrose inverse of an operator as $\alpha \downarrow 0$. For noisy data $\Vert y^{\delta }-y\Vert \le \delta$, a convergent regularization of an inverse problem also specifies a parameter choice $\alpha (\delta ,y^{\delta })$ giving convergence to $A^{†}y$, uniformly over the allowed noise at each fixed admissible exact datum.

Expand $x_{\alpha }$ in the input singular vectors. The Tikhonov normal equation impliesThe component in $\ker A$ is zero because $\alpha >0$. Comparing with the given filter convention givesThe singular-system Tikhonov filter is often described by its gain $g_{\alpha }(\sigma )=\sigma /(\alpha +{\sigma }^2)$; the requested $f_{\alpha }$ includes one extra factor of $\sigma$. For admissible data, $f_{\alpha }\to 1$ and $0\le f_{\alpha }\le 1$, so dominated convergence theorem applied to the squared Picard criterion coefficients proves exact-data convergence. AlsoBy the noise-bias decomposition for linear regularization, $\alpha (\delta )\to 0$ and $\delta /\sqrt{\alpha (\delta )}\to 0$ suffice for noisy-data convergence; for example $\alpha =\delta$ as $\delta \downarrow 0$. This establishes both consistency and stability of Tikhonov regularization.

A linear regularization is a family of bounded linear operators $R_{\alpha }:V\to U$, $\alpha >0$, for which $R_{\alpha }f\to K^{†}f$ for every $f\in \mathcal{D}(K^{†})$ as $\alpha \downarrow 0$. To obtain a convergent regularization of an inverse problem for noisy data, choose a regularization parameter so that the exact-data approximation error and the amplified noise both vanish.

An example is Tikhonov regularization:The positive quadratic term makes $K^{\ast }K+\alpha I$ invertible. The Tikhonov filter norm bound follows from the filter $s/(s^2+\alpha )$, whose maximum for $s\ge 0$ is $1/(2\sqrt{\alpha })$. Therefore $\Vert R_{\alpha }\Vert \le 1/(2\sqrt{\alpha })$. The spectral factors $s^2/(s^2+\alpha )$ tend to one on the positive spectrum; dominated convergence theorem gives consistency on $\mathcal{D}(K^{†})$. The noise-bias decomposition for linear regularization then proves convergence whenever $\alpha (\delta )\to 0$ and $\delta /\sqrt{\alpha (\delta )}\to 0$, for example $\alpha (\delta )=\delta$ for sufficiently small positive noise levels.

The Volterra integration operator is injective and has dense range. Its range of the Volterra integration operator is $\{f\in H^1(0,1):f(0)=0\}$, and its Moore–Penrose inverse of an operator is differentiation on this domain. Thus the error estimate for $K^{†}f$ implicitly requires $f(0)=0$, in addition to the printed $H^2$ assumption. For an arbitrary $H^2$ function the same calculation bounds error against $f^{\prime }$, but that function need not be in $\mathcal{D}(K^{†})$.

Put $a=(1-\alpha )/2$, $b=(1+\alpha )/2$ and $h=\alpha /2$. The three stencils are forward, central finite differences, and backward on $[0,a)$, $[a,b)$ and $[b,1]$, respectively. All shifted evaluation points lie in $[0,1]$. For $g\in L^2(0,1)$, apply $|A-B{|}^2\le 2(|A{|}^2+|B{|}^2)$ and change variables in each of the six evaluation terms. The resulting integration intervals arePair $[0,a]$ with $[a,1-\alpha ]$, $[\alpha ,b]$ with $[b,1]$, and the two middle intervals. Each pair has disjoint interiors, so almost every point is counted at most three times. This overlap multiplicity bound for piecewise difference operators provesFor the bias on the middle interval, write the central finite difference as an average of $f^{\prime }$. The fundamental theorem of calculus giveswhere $\Vert k_{\alpha }{\Vert }_1=\alpha /4$. Extend $f^{\prime \prime }$ by zero to the real line. Young's convolution inequality bounds the middle-interval $L^2$ error by $\alpha c/4$. Combining its squared error with the supplied outer-interval estimate gives the central-difference noise-bias boundThe triangle inequality and the operator norm estimate now giveFor $c>0$, balancing noise and approximation bias minimizes this upper bound atfor sufficiently small $\delta$ that $\alpha <1/2$. Capping the parameter at $1/4$ supplies an admissible rule for all positive noise levels. More generally, $\alpha \to 0$ and $\delta /\alpha \to 0$ suffice; $\alpha =\sqrt{\delta }$ avoids needing $c$ or dividing by $c=0$.

To prove convergence for every admissible datum, not just $H^2$ data, let $f=Ku$ with $u\in L^2(0,1)$ and extend $u$ by zero. Each stencil is a forward, centred or backward average of $u$. Translation continuity in Lp makes each averaged difference from $u$ tend to zero in $L^2(\mathbb{R})$. Restricting those three bounds to their respective intervals proves $R_{\alpha }Ku\to u$. The noise-bias decomposition for linear regularization, with $\delta /\alpha \to 0$, finishes the convergent regularization of an inverse problem.

For a linear regularization, add and subtract the exact-data reconstruction:The triangle inequality gives the noise-bias decomposition for linear regularization:The two summands are noise amplification and approximation bias. Increasing the strength of regularization typically decreases noise amplification and increases bias; reducing regularization has the opposite effect. These are qualitative tendencies and bounds, not a claim that each realized noise error is monotone for every noise vector.

For the one-sided differentiation example, the bounds are $\sqrt{6}\delta /h$ and $\Vert f^{\prime \prime }{\Vert }_{\infty }h/2$. Their sum has an interior balance point when the second-derivative bound is positive and the optimum lies in the permitted interval. The following original sketch plots these bounds against $h$:

The approximate source bound for coercive Tikhonov regularization first allows a fixed sufficiently large $r$ to make ${\eta }_r$ small, then a sufficiently small $\alpha$ to make $\sqrt{\alpha }\beta r$ small. This order of limits proves $R_{\alpha }f\to K^{†}f$ for every datum in its domain.

For the noise error let $v=R_{\alpha }g$. Its normal equation for coercive Tikhonov regularization givesCompleting the square shows $\alpha {\beta }^2\Vert v{\Vert }^2\le \Vert g{\Vert }^2/4$, hence $\Vert R_{\alpha }\Vert \le 1/(2\beta \sqrt{\alpha })$. The noise-bias decomposition for linear regularization now givesTherefore the sufficient regularization parameter choice isFor example $\alpha (\delta )={\delta }^q$ with $0<q<2$ works. Coercivity makes $K^{\ast }K+\alpha B^{\ast }B$ boundedly invertible for each positive $\alpha$, so these are continuous linear reconstruction operators and form a convergent regularization of an inverse problem.

Suppose each $R_{\alpha }:\mathcal{V}\to \mathcal{U}$ is a bounded linear operator and $R_{\alpha }f\to K^{†}f$ for every $f\in \mathcal{D}(K^{†})$ as $\alpha \downarrow 0$. A standard sufficient a priori regularization parameter choice satisfiesFor every datum with $\Vert f^{\delta }-f\Vert \le \delta$, the noise-bias decomposition for linear regularization givesThus the parameter rule gives a convergent regularization of an inverse problem, uniformly over data in the prescribed noise ball for each fixed admissible exact datum. For Tikhonov regularization, $\Vert R_{\alpha }\Vert \le 1/(2\sqrt{\alpha })$, so $\alpha \to 0$ and $\delta /\sqrt{\alpha }\to 0$ suffice. The same sufficient noise scaling holds for spectral cutoff regularization.

Write $y={\psi }_s$ and retain the Landweber relaxation parameter $\tau$. The printed $\Vert A\Vert <2$ allows the fixed choice $\tau =1/2$, since then $\tau \Vert A{\Vert }^2<2$. It does not justify a unit step: the scalar linear operator $A=3/2$ has norm below $2$, but unit-step error is multiplied by $1-|A{|}^2=-5/4$ and diverges. We use $0<\tau <2/\Vert A{\Vert }^2$ throughout.

From the recurrence and $f_0=0$, induction yields the closed formFor the compact operator in part ii, take a singular system of a compact operator with $A{v}_j={\sigma }_j{u}_j$, $A^{\ast }{u}_j={\sigma }_j{v}_j$ and ${\sigma }_j>0$. Thus ${\sigma }_j^2$ are the eigenvalues of $A^{\ast }A$; this fixes the notation implicit in the printed hint. In the singular component $j$, the recurrence isSumming the geometric series gives the Landweber spectral filterThere is no contribution from data in $\ker A^{\ast }$ or from $\ker A$ in the reconstruction. Starting at zero is what selects the minimum-norm least-squares solution.

For fixed ${\sigma }_j>0$, $|1-\tau {\sigma }_j^2|<1$, so the numerator tends to $1$. The assumption $y\in D(A^{†})$ is the Picard criterionwith an arbitrary additional component in $\ker A^{\ast }$. The squared reconstruction error isEach term tends to zero and is bounded by a summable term from the Picard criterion. The dominated convergence theorem therefore proves $R_{n}y\to A^{†}y$, where $A^{†}$ is the Moore–Penrose inverse of an operator.

For each finite $n$, the regularization of an inverse problem is stable. Set $t=\tau {\sigma }^2\in (0,2)$. The geometric series and $|1-t|\le 1$ implyConsequently the Landweber noise amplification bound in this general step-size convention is $\Vert R_n\Vert \le \sqrt{2\tau n}$. If additionally $\tau \Vert A{\Vert }^2\le 1$, the sharper numerator bound $\min (nt,1)$ gives $\Vert R_n\Vert \le \sqrt{\tau n}$.

To see the regularization parameter directly, put $\alpha =1/(\tau n)$. Modes with ${\sigma }^2\ll \alpha$ have numerator approximately $n\tau {\sigma }^2$, so their inverse coefficient is approximately $\sigma /\alpha$ rather than $1/\sigma$. Each fixed nonzero mode is eventually restored as $\alpha \downarrow 0$. ThusThe finite iterates suppress unstable small singular values; infinitely many iterations remove this suppression. For noisy data $\Vert y^{\delta }-y\Vert \le \delta$, the noise-bias decomposition for linear regularization givesChoosing $n(\delta )\to \infty$ and ${\delta }^{2}n(\delta )\to 0$ proves noisy-data convergence. The reciprocal iteration index is a regularization parameter, and early stopping controls noise amplification.

Let $u^{†}=A^{†}f$. Linearity and the triangle inequality give the noise-bias decomposition for linear regularizationThe first term tends to zero by the assumed noise-amplification bound. The second tends to zero because $\alpha (\delta )\to 0$ and $R_{\alpha }$ is a regularization of an inverse problem. The right side is independent of the particular noisy datum within its allowed ball. Its convergence therefore proves the uniform noisy-data convergence required of a convergent regularization of an inverse problem.

Add and subtract the exact-data iterate. The noise-bias decomposition for linear regularization givesso the supplied Landweber noise amplification bound yieldsWith unit step and zero initial iterate, the exact-data error is $x^{†}-x_n=(I-A^{\ast }A{)}^n{x}^{†}$. Its squared norm isby the dominated convergence theorem. Stopping with $n\to \infty$ but $n{\delta }^2\to 0$ therefore makes both errors vanish.

Regularization replaces sensitive inversion by a controlled approximation. For a bounded linear operator between Hilbert spaces, a linear regularization is a family of bounded linear operators $R_{\alpha }$ satisfying $R_{\alpha }y\to A^{†}y$ as $\alpha \downarrow 0$ for each datum in the domain of the Moore–Penrose inverse of an operator. Noisy-data convergence additionally requires a suitable regularization parameter choice.^[ref] Nonlinear variational regularization can instead use nonquadratic penalties.

The noise-bias decomposition for linear regularization separates two costs: amplifying measurement noise and changing the exact-data reconstruction. Truncated singular value decomposition discards weak modes; Tikhonov regularization attenuates them smoothly. A quadratic norm penalty expresses a preference for smaller reconstructions. Neither method manufactures a missing null space component: with complete averaging, the minimum-norm least-squares solution for $(0.5,0.5{)}^T$ is $(0.5,0.5{)}^T$, even if the original image was $(1,0{)}^T$. The preference selects a representative; extra observations or justified constraints are needed to identify the original.

For Tikhonov regularization, the Tikhonov stability bound limits transmitted measurement noise of norm at most $\delta$ to $\delta /(2\sqrt{\alpha })$. Taking $\alpha \to 0$ while $\delta /\sqrt{\alpha }\to 0$ gives a sufficient rule for convergent regularization of an inverse problem. Holding $\alpha$ fixed leaves approximation bias; letting it vanish too quickly can admit overwhelming measurement noise.

For a bounded linear operator between Hilbert spaces, the spectral regularization method for quadratic Tikhonov regularization has scalar coefficient $s/(s^2+\alpha )$. Since $(s-\sqrt{\alpha }{)}^2\ge 0$, this coefficient is at most $1/(2\sqrt{\alpha })$. The spectral theorem for normal operators transfers this scalar bound to the operator norm, providing the noise term in the noise-bias decomposition for linear regularization.