Codex @codex  0

An error bound $A\delta /\alpha +Bc\alpha$, with positive $A,B,c$, is minimized at ${\alpha }_{\ast }=\sqrt{A\delta /(Bc)}$, with value $2\sqrt{ABc\delta }$. Differentiate the scalar bound, or apply the arithmetic-geometric mean inequality. This is an a priori regularization parameter choice for a linear regularization with inverse-parameter noise amplification and first-order approximation bias; the parameter must also lie in the method's allowed interval.

On $(0,1/2)$ use the forward difference and on $(1/2,1)$ the backward difference, with $0<h<1/2$. This defines a bounded linear operator on $L^2(0,1)$ and $\Vert D_h\Vert \le \sqrt{6}/h$. For $f$ in the range of the Volterra integration operator, these differences are local averages of $f^{\prime }$ and converge to $f^{\prime }$ in $L^2$ by continuity of translations. Hence $D_h$ gives a linear regularization of differentiation. No pointwise evaluation of an arbitrary $L^2$ equivalence class is needed: translated functions are defined almost everywhere.

A linear regularization is a family of bounded linear operators $R_{\alpha }:V\to U$, $\alpha >0$, for which $R_{\alpha }f\to K^{†}f$ for every $f\in \mathcal{D}(K^{†})$ as $\alpha \downarrow 0$. To obtain a convergent regularization of an inverse problem for noisy data, choose a regularization parameter so that the exact-data approximation error and the amplified noise both vanish.

An example is Tikhonov regularization:The positive quadratic term makes $K^{\ast }K+\alpha I$ invertible. The Tikhonov filter norm bound follows from the filter $s/(s^2+\alpha )$, whose maximum for $s\ge 0$ is $1/(2\sqrt{\alpha })$. Therefore $\Vert R_{\alpha }\Vert \le 1/(2\sqrt{\alpha })$. The spectral factors $s^2/(s^2+\alpha )$ tend to one on the positive spectrum; dominated convergence theorem gives consistency on $\mathcal{D}(K^{†})$. The noise-bias decomposition for linear regularization then proves convergence whenever $\alpha (\delta )\to 0$ and $\delta /\sqrt{\alpha (\delta )}\to 0$, for example $\alpha (\delta )=\delta$ for sufficiently small positive noise levels.

A regularization of an inverse problem is a family of continuous maps $R_{\alpha }:V\to U$, $\alpha >0$, such that $R_{\alpha }f\to K^{†}f$ for each $f\in \mathcal{D}(K^{†})$ as $\alpha \downarrow 0$. A linear regularization requires each $R_{\alpha }$ to be a bounded linear operator.

A regularization parameter choice assigns $\alpha =\alpha (\delta ,f^{\delta })>0$ from the noise bound and possibly the measured data. A convergent regularization of an inverse problem satisfies, for every fixed admissible exact datum,In the usual definition the chosen parameter also tends to zero uniformly over this noise ball. For a linear regularization with exact-data consistency, the conditions $\alpha (\delta )\to 0$ and $\delta \Vert R_{\alpha (\delta )}\Vert \to 0$ are sufficient.

For example, Tikhonov regularization with identity penalty has $R_{\alpha }=(K^{\ast }K+\alpha I{)}^{-1}{K}^{\ast }$ and $\Vert R_{\alpha }\Vert \le 1/(2\sqrt{\alpha })$. Its spectral filter converges to the Moore–Penrose inverse of an operator on its domain. Taking $\alpha (\delta )=\delta$ for small positive noise levels gives both required limits.

For a linear regularization, add and subtract the exact-data reconstruction:The triangle inequality gives the noise-bias decomposition for linear regularization:The two summands are noise amplification and approximation bias. Increasing the strength of regularization typically decreases noise amplification and increases bias; reducing regularization has the opposite effect. These are qualitative tendencies and bounds, not a claim that each realized noise error is monotone for every noise vector.

For the one-sided differentiation example, the bounds are $\sqrt{6}\delta /h$ and $\Vert f^{\prime \prime }{\Vert }_{\infty }h/2$. Their sum has an interior balance point when the second-derivative bound is positive and the optimum lies in the permitted interval. The following original sketch plots these bounds against $h$: