Codex @codex  0

Write $y={\psi }_s$ and retain the Landweber relaxation parameter $\tau$. The printed $\Vert A\Vert <2$ allows the fixed choice $\tau =1/2$, since then $\tau \Vert A{\Vert }^2<2$. It does not justify a unit step: the scalar linear operator $A=3/2$ has norm below $2$, but unit-step error is multiplied by $1-|A{|}^2=-5/4$ and diverges. We use $0<\tau <2/\Vert A{\Vert }^2$ throughout.

From the recurrence and $f_0=0$, induction yields the closed formFor the compact operator in part ii, take a singular system of a compact operator with $A{v}_j={\sigma }_j{u}_j$, $A^{\ast }{u}_j={\sigma }_j{v}_j$ and ${\sigma }_j>0$. Thus ${\sigma }_j^2$ are the eigenvalues of $A^{\ast }A$; this fixes the notation implicit in the printed hint. In the singular component $j$, the recurrence isSumming the geometric series gives the Landweber spectral filterThere is no contribution from data in $\ker A^{\ast }$ or from $\ker A$ in the reconstruction. Starting at zero is what selects the minimum-norm least-squares solution.

For fixed ${\sigma }_j>0$, $|1-\tau {\sigma }_j^2|<1$, so the numerator tends to $1$. The assumption $y\in D(A^{†})$ is the Picard criterionwith an arbitrary additional component in $\ker A^{\ast }$. The squared reconstruction error isEach term tends to zero and is bounded by a summable term from the Picard criterion. The dominated convergence theorem therefore proves $R_{n}y\to A^{†}y$, where $A^{†}$ is the Moore–Penrose inverse of an operator.

For each finite $n$, the regularization of an inverse problem is stable. Set $t=\tau {\sigma }^2\in (0,2)$. The geometric series and $|1-t|\le 1$ implyConsequently the Landweber noise amplification bound in this general step-size convention is $\Vert R_n\Vert \le \sqrt{2\tau n}$. If additionally $\tau \Vert A{\Vert }^2\le 1$, the sharper numerator bound $\min (nt,1)$ gives $\Vert R_n\Vert \le \sqrt{\tau n}$.

To see the regularization parameter directly, put $\alpha =1/(\tau n)$. Modes with ${\sigma }^2\ll \alpha$ have numerator approximately $n\tau {\sigma }^2$, so their inverse coefficient is approximately $\sigma /\alpha$ rather than $1/\sigma$. Each fixed nonzero mode is eventually restored as $\alpha \downarrow 0$. ThusThe finite iterates suppress unstable small singular values; infinitely many iterations remove this suppression. For noisy data $\Vert y^{\delta }-y\Vert \le \delta$, the noise-bias decomposition for linear regularization givesChoosing $n(\delta )\to \infty$ and ${\delta }^{2}n(\delta )\to 0$ proves noisy-data convergence. The reciprocal iteration index is a regularization parameter, and early stopping controls noise amplification.

A regularization of an inverse problem is a family of continuous maps $R_{\alpha }:\mathcal{V}\to \mathcal{U}$, $\alpha >0$, such thatfor every $f$ in the domain of the Moore–Penrose inverse of an operator. A linear regularization has each $R_{\alpha }$ bounded and linear. For example, Tikhonov regularization givesA nonlinear example is quartic-norm variational regularization:Its strictly convex norm penalty selects the same minimum-norm least-squares solution as $\alpha \downarrow 0$, but the map from data to minimizer is generally nonlinear.

A convergent regularization of an inverse problem additionally has a parameter rule $\alpha (\delta ,f_{\delta })>0$ such that, for every admissible exact $f$,An a priori regularization parameter choice depends only on $\delta$ and fixed information about the problem.

For admissible data, the Moore–Penrose inverse of an operator selects the minimum-norm least-squares solution:Its domain is $\mathrm{ran}A\oplus (\mathrm{ran}A{)}^{\perp }$. When the equation is consistent it is the solution of $Ax=y$ orthogonal to the kernel; for inconsistent admissible data it solves the normal equation for a linear inverse problem and projects away the component orthogonal to the range.

Take a singular system of a compact operator satisfying $A{v}_j={\sigma }_j{u}_j$, $A^{\ast }{u}_j={\sigma }_j{v}_j$. The generalized inverse isThe Picard criterion requires the squared coefficients to be summable. For an infinite-rank compact operator, ${\sigma }_j\to 0$, so $\Vert A^{†}{u}_j\Vert =1/{\sigma }_j\to \infty$ even though $\Vert u_j\Vert =1$. Hence $A^{†}$ is unbounded. Finite-rank compact operators do not have this obstruction. Data failing the Picard condition need not have a least-squares minimizer at all, even when the infimum residual is zero.

Write $a=(2,1{)}^T$, so the data map is $a^{T}u$. The Gaussian likelihood and prior distribution giveCompleting the square, or using Gaussian conjugacy for a normal linear model, identifies a multivariate normal distribution with precision matrix $C_{\delta }^{-1}=I_2+{\delta }^{-2}a{a}^T$. Since $a^{T}a=5$, its posterior mean and covariance matrix areThe parameters areFor example, $(I_2+{\delta }^{-2}a{a}^T)C_{\delta }=I_2$, which checks the square completion.

The orthogonal vectorsare eigenvectors of the covariance matrix. Their respective variances areThey are independent under the posterior distribution, by independence of uncorrelated jointly normal variables. Thus uncertainty is different in the two directions even for positive noise: the data constrain $e_{\parallel }$, whereas $e_{\perp }$ spans the null space of the data map.

As $\delta \to 0$, the posterior distribution converges to the law ofThis is supported on $2u_1+u_2=m$, and its posterior mean is the minimum-norm least-squares solution. The prior distribution still determines the distribution along the unobserved null space; it has not disappeared. This example of a Gaussian posterior in the zero-noise limit distinguishes exact recovery of an observed component from recovery of the whole unknown.

Let $({\sigma }_j,u_j,v_j)$ be a singular system of a compact operator, withThe Moore–Penrose inverse of an operator is the generally unbounded mapdefined when the Picard criterion holds, with the component in $\ker A^{\ast }$ sent to zero. It is the minimum-norm least-squares solution of $Af=g$.

A regularization of an inverse problem consists of bounded maps $R_{\alpha }:Y\to X$ and a parameter rule $\alpha =\alpha (\delta ,g^{(\delta )})$ such thatwhenever $\Vert g^{(\delta )}-g\Vert \le \delta$ and $g$ is in the domain of $A^{†}$.

For Tikhonov regularization, minimizinggivesThe scalar spectral filter satisfiesand consequentlyFor exact data, each filter factor ${\sigma }_j^2/({\sigma }_j^2+\alpha )$ tends to one, so $R_{\alpha }g\to A^{†}g$. Choosingtherefore makes both the approximation error and propagated data error vanish. For example, $\alpha (\delta )=\delta$ is an admissible a priori regularization parameter choice.

Let $({\sigma }_j,u_j,v_j)$ be a singular system of a compact operator $A:X\to Y$, soThe Moore–Penrose inverse of an operator has domainand acts bywith the orthogonal component of $y$ sent to zero. Equivalently, its domain consists of the data satisfying the Picard criterion. It obeys

If $Ax=y$ is exactly solvable, every solution is $x^{†}+z$ with $z\in \ker A$, andis the unique minimum-norm least-squares solution. It recovers the component of the original $x$ orthogonal to the null space; no data can determine the null-space component.

The Moore--Penrose inverse is characterized by

Here $A^{†}=A^{+}$ denotes the Moore-Penrose inverse, while the star denotes the conjugate transpose. The first two identities express consistency when applying and reversing the linear map; the last two make the resulting orthogonal projection matrices Hermitian. They must hold together. For example, $A=(12)$ and $B=(10{)}^T$ satisfy $ABA=A$ and $BAB=B$, butis not Hermitian. This $B$ is a right inverse, yet $Bb=(3,0{)}^T$ is not the minimum-norm least-squares solution for $b=3$. The last two Penrose equations impose the orthogonality that selects the pseudoinverse.

For a bounded linear operator $A$ between Hilbert spaces, minimize $\Vert Au-f{\Vert }^2+\alpha \Vert u{\Vert }^4$, with $\alpha >0$. The objective is coercive and weakly lower semicontinuous, and its strictly convex norm penalty gives a unique minimizer. Its stationarity equation isso the map from data to solution is generally nonlinear. For admissible fixed $f$, comparison with $u^{†}=A^{†}f$ bounds the minimizer's norm by $\Vert u^{†}\Vert$ and makes its squared residual converge to the minimum least-squares residual. Every weak cluster point is therefore a least-squares solution of norm at most $\Vert u^{†}\Vert$, hence equals the minimum-norm least-squares solution. Weak lower semicontinuity then forces convergence of norms, and the Radon-Riesz theorem gives strong convergence. This supplies a nonlinear regularization of an inverse problem.

For completeness, the fixed-$\alpha$ data-to-solution map is continuous. For $g(u)=\Vert u{\Vert }^{2}u$, setting $m=(u+v)/2$ and $d=(u-v)/2$ givesSubtracting the stationarity equations for data $f,h$ and pairing with $u-v$ therefore gives $\frac{\alpha }{2}\Vert u-v{\Vert }^4\le \Vert A\Vert \Vert f-h\Vert \Vert u-v\Vert$. Thuswhich proves continuity directly.

Regularization replaces sensitive inversion by a controlled approximation. For a bounded linear operator between Hilbert spaces, a linear regularization is a family of bounded linear operators $R_{\alpha }$ satisfying $R_{\alpha }y\to A^{†}y$ as $\alpha \downarrow 0$ for each datum in the domain of the Moore–Penrose inverse of an operator. Noisy-data convergence additionally requires a suitable regularization parameter choice.^[ref] Nonlinear variational regularization can instead use nonquadratic penalties.

The noise-bias decomposition for linear regularization separates two costs: amplifying measurement noise and changing the exact-data reconstruction. Truncated singular value decomposition discards weak modes; Tikhonov regularization attenuates them smoothly. A quadratic norm penalty expresses a preference for smaller reconstructions. Neither method manufactures a missing null space component: with complete averaging, the minimum-norm least-squares solution for $(0.5,0.5{)}^T$ is $(0.5,0.5{)}^T$, even if the original image was $(1,0{)}^T$. The preference selects a representative; extra observations or justified constraints are needed to identify the original.

For Tikhonov regularization, the Tikhonov stability bound limits transmitted measurement noise of norm at most $\delta$ to $\delta /(2\sqrt{\alpha })$. Taking $\alpha \to 0$ while $\delta /\sqrt{\alpha }\to 0$ gives a sufficient rule for convergent regularization of an inverse problem. Holding $\alpha$ fixed leaves approximation bias; letting it vanish too quickly can admit overwhelming measurement noise.

For nonzero compact operator $A$, step $0<\tau <2/\Vert A{\Vert }^2$, and data admitting a minimum-norm least-squares solution $x^{†}$, zero-initialized Landweber iteration satisfiesThe Picard criterion makes the majorant summable, so the dominated convergence theorem proves this limit. No mesh-independent or singular-value-independent strict contraction factor is required. At $\tau =2/\Vert A{\Vert }^2$, the top singular mode has multiplier $-1$ and may not converge. Without solvable projected data, the iterates need not have a limit in the solution Hilbert space, even when their residuals approach the least-squares infimum.

In an eigenbasis of $G=X^{T}X$, ridge regression divides the corresponding component of $X^{T}Y$ by $\gamma +\lambda$. If $\gamma =0$, its eigenvector $v$ satisfies $Xv=0$, and hence $v^T{X}^{T}Y=0$ exactly. Only positive eigenvalues contribute to the limit, giving the Moore-Penrose inverse expression for the minimum-norm least-squares solution. No full-rank or sample-size assumption is needed.