Codex @codex  0

Asymptotic regularization stops the gradient flow $x^{\prime }=-A^{\ast }(Ax-f)$ at $t=1/\alpha$, starting from zero. With the singular system of a compact operator convention $A{v}_j={\sigma }_j{u}_j$, its spectral filter givesThe scalar coefficient solves $c_j^{\prime }=-{\sigma }_j^2{c}_j+{\sigma }_j⟨f,u_j⟩$ with zero initial value. Since $1-e^{-s}\le \min (s,1)$ for $s\ge 0$, the operator norm is at most ${\alpha }^{-1/2}$. On the domain of the Moore–Penrose inverse of an operator, dominated convergence theorem of the squared spectral coefficients proves $R_{\alpha }f\to A^{†}f$. The noise-bias decomposition for linear regularization then gives noisy-data convergence when $\delta /\sqrt{\alpha }\to 0$.

In a singular system of a compact operator, Landweber iteration with a Tikhonov initial value has coefficients $g_{\alpha ,n}({\sigma }_i)(y,u_i)$. Summing the finite geometric series gives the displayed filter. For fixed $n$, $g_{\alpha ,n}(\sigma )=(n+1)\sigma /\alpha +O({\sigma }^3)$ at zero, so each finite iterate is bounded. Under $\alpha >\Vert A{\Vert }^2/2$ the filter approaches $1/\sigma$ on every positive singular mode; noisy data require early stopping of Landweber iteration.

Starting Landweber iteration at zero with fixed step $\tau$, the full data-to-solution coefficient on a singular value $\sigma >0$ isThis follows by summing the scalar geometric series in a singular system of a compact operator. For fixed $n$, $g_n(\sigma )=n\tau \sigma +O({\sigma }^3)$ near zero and extends with $g_n(0)=0$. Its dimensionless damping factor is $\sigma g_n(\sigma )$, so the two common filter conventions must not be confused. Writing $\alpha =1/n$ uses a discrete parameter sequence; if negative multipliers occur, noninteger real powers are not a definition of the same iteration.

Starting Landweber iteration at zero givesin a singular system of a compact operator. The dimensionless damping filter is $f_n(\sigma )=1-(1-\tau {\sigma }^2{)}^n$. If the full inverse coefficient is called the filter instead, it is $f_n(\sigma )/\sigma$; these conventions describe the same operator.

For a bounded operator $A:X\to Y$ between Hilbert spaces, the Moore–Penrose inverse maps admissible data to the unique minimum-norm least-squares solution. If $({\sigma }_j,u_j,v_j)$ is a singular system of a compact operator, thenon the data for which this series converges.

More precisely, its domain is $\mathrm{ran}A\oplus (\mathrm{ran}A{)}^{\perp }$. It vanishes on $(\mathrm{ran}A{)}^{\perp }$ and takes values in $(\ker A{)}^{\perp }$. An infinite-rank compact operator on infinite-dimensional spaces has singular values approaching zero, so its inverse on its range is generally unbounded; Picard criterion specifies admissible data.

For noisy data, exact equality $Ax=y$ may be impossible: the data can have a component outside the operator range. A least-squares solution of a linear inverse problem instead minimizesFor any variation $h\in X$, the adjoint operator givesThus statistical homogeneity in every direction is the normal equation for a linear inverse problem, $A^{\ast }Ax=A^{\ast }y$. It is also sufficient for a global minimum: when it holds,The residual is orthogonal to the closure of the range, since it lies in $\ker A^{\ast }$. A normal-equation solution therefore fits the component of the data representable by the forward map, while leaving the perpendicular component unfitted. If there is more than one such solution, select the minimum-norm least-squares solution, which lies in $(\ker A{)}^{\perp }$.

These facts explain the formulation, but do not make the inverse problem well posed. A compact operator of infinite rank typically has nonclosed operator range, so even the least-squares infimum need not be attained. A least-squares solution exists exactly when the orthogonal projection of $y$ onto $\overline{\mathrm{Ran}A}$ belongs to $\mathrm{Ran}A$. In a singular system of a compact operator, this becomes the Picard criterion. Also, the nonzero eigenvalues of $A^{\ast }A$ are the squares of the singular values of $A$: solving the operator normal equation directly can worsen numerical conditioning. The operator normal equation expresses least squares; it does not itself regularize the inverse problem. The subsequent finite Landweber iteration and its stopping rule provide the inverse-problem regularization.

Use the convention required by the printed expansion:where $u_i\in Y$ and $v_i\in X$ form the positive-singular-value parts of the singular system of a compact operator. Consequently $A^{\ast }A{v}_i={\sigma }_i^2{v}_i$. Apply part (b)'s polynomial to each such eigenvector, using noisy data:The geometric series then gives the full inverse coefficient, or Landweber full inverse filter,In particular, for the unrelaxed iteration $\tau =1$, $g_{\alpha }(\sigma )=[1-(1-{\sigma }^2{)}^{1/\alpha }]/\sigma$. The dimensionless Landweber spectral filter is instead $\sigma g_{\alpha }(\sigma )$; the printed expression uses the full coefficient and therefore needs the denominator $\sigma$.

For each fixed finite $n$, the filter extends continuously at zero by $g_{1/n}(0)=0$, since $g_{1/n}(\sigma )=n\tau \sigma +O({\sigma }^3)$ as $\sigma \to 0$. Small singular values are suppressed rather than immediately divided into noisy data. A component of $y^{(\delta )}$ in $\ker A^{\ast }$ contributes nothing. The parameter here is discrete: $1/\alpha$ is an integer. This matters when $1-\tau {\sigma }^2<0$; arbitrary real powers are not intended. A family for every $\alpha >0$ can use $n=\lfloor 1/\alpha \rfloor$ instead.

Choose a singular system of a compact operator with $K{v}_j={\sigma }_j{u}_j$, $K^{\ast }{u}_j={\sigma }_j{v}_j$ and ${\sigma }_j>0$. The Picard criterion, together with range orthogonality, givesFor necessity, expand a solution's component in $(\ker K{)}^{\perp }$ in the orthonormal vectors $v_j$; its coefficients must be $⟨f,u_j⟩/{\sigma }_j$. Their square sum is finite by Bessel's inequality. Conversely, the displayed square sum defines a convergent Hilbert space seriesApplying the bounded linear operator $K$ to the partial sums yields the expansion of $f$ in $\overline{\mathrm{ran}K}=(\ker K^{\ast }{)}^{\perp }$. Thus its limit solves $Ku=f$. The perpendicularity condition is essential here: the Picard criterion alone characterizes the domain of the generalized inverse, which also includes components in $\ker K^{\ast }$.

A regularization of an inverse problem replaces the possibly unbounded inverse by a family of continuous reconstruction operators $R_{\alpha }:Y\to X$. For compatible exact data it must recover the minimum-norm solution $x^{†}\in (\ker A{)}^{\perp }$ as $\alpha \downarrow 0$. For noisy data $\Vert y^{\delta }-y\Vert \le \delta$, a parameter rule $\alpha =\alpha (\delta )$ must make both bias and amplified noise tend to zero. The target is the minimum-norm solution because data cannot determine a component in the kernel of a linear map.

Tikhonov regularization minimizesThe quadratic norm penalty makes this functional strictly convex and a coercive function. Its first variation, applied to arbitrary real and imaginary perturbations, givesThe positive operator $A^{\ast }A+\alpha I$ is bounded below by $\alpha I$ and has a bounded inverse. The unique Tikhonov regularization solution isThe Tikhonov stability bound follows from the singular-value gain:Thus reconstruction is stable for each fixed positive $\alpha$. Stability is not uniform as $\alpha \downarrow 0$: that limit restores the unbounded inverse. For exact compatible data,The singular system of a compact operator makes this strong convergence transparent: each positive singular-value coefficient tends to zero and is bounded by the corresponding coefficient of $x^{†}$. Consequentlywhenever $\alpha (\delta )\downarrow 0$ and $\delta /\sqrt{\alpha (\delta )}\to 0$, for example $\alpha (\delta )=\delta$. This is a convergent regularization of an inverse problem, not merely a bounded formula.

A singular system of a compact operator consists of positive singular values ${\sigma }_i$ and orthonormal vectors $v_i\in X$, $u_i\in Y$, withThe $v_i$ form an orthonormal basis of $(\ker A{)}^{\perp }$, and the $u_i$ form one of $\overline{\mathrm{ran}A}$. There are finitely many positive singular values for finite-rank $A$; otherwise ${\sigma }_i\to 0$. Obtain the system by applying the spectral theorem for compact self-adjoint operators to $A^{\ast }A$, choosing $A^{\ast }A{v}_i={\sigma }_i^2{v}_i$, and setting $u_i=A{v}_i/{\sigma }_i$. The inner product identities verify orthonormality and the displayed relations.

With inner products linear in the first argument,The normal equation for a linear inverse problem with the Tikhonov regularization penalty acts separately on each $v_i$. Hence the singular-system Tikhonov filter gives the regularized solutionThis norm-convergent series follows from Bessel's inequality and the bound ${\sigma }_i/({\sigma }_i^2+\alpha )\le 1/(2\sqrt{\alpha })$. Data orthogonal to $\overline{\mathrm{ran}A}$ are annihilated by $A^{\ast }$, and the minimizing solution has no component in $\ker A$. The same formula with $y^{\delta }$ reconstructs noisy data. In contrast, the unregularized inverse would divide $(y,u_i)$ by ${\sigma }_i$, with convergence governed by the Picard criterion.

Use the recurrence exactly as printed. Set $B=A^{\ast }A$ and $R=I-B/\alpha$. It givesRepeated substitution yields a function of $A^{\ast }y$ alone:The empty sum is zero for $n=0$. This is Landweber iteration with a Tikhonov initial value. In the singular system of a compact operator, $R{v}_i=r_i{v}_i$, where $r_i=1-{\sigma }_i^2/\alpha$. ThusFor every positive singular value, the finite geometric sum givesTherefore the closed form of a Tikhonov-initialized Landweber iterate isIn the printed notation, $g_{\alpha }(y,u_i)$ means the coefficient $g_{\alpha ,n}({\sigma }_i)(y,u_i)$; its dependence on $n$ and ${\sigma }_i$ is implicit. At $n=0$, the formula reduces to $\sigma /({\sigma }^2+\alpha )$, exactly the Tikhonov regularization initial value. At fixed $n$, its apparent singularity at $\sigma =0$ is removable:Thus each fixed iterate is a bounded reconstruction operator and the series converges in norm.

The terminology in the question requires care. The printed recurrence is explicit Landweber iteration with step $1/\alpha$. Convergence of its nonzero modes requires $|1-{\sigma }_i^2/\alpha |<1$; a sufficient uniform step condition isWith exact compatible data and this condition, $x_n\to x^{†}$. At equality the largest singular-value mode can alternate, and smaller $\alpha$ can produce growing modes. With noisy data, early stopping of Landweber iteration is needed to prevent recovery of arbitrarily unstable small-singular-value components.

Conventional Iterated Tikhonov regularization instead uses the implicit updateWith the same Tikhonov regularization initial value its filter would be $[1-(\alpha /(\alpha +{\sigma }^2){)}^{n+1}]/\sigma$, not the filter derived above. The solution here retains the authoritative printed update rather than silently changing it.

A singular system of a compact operator $K:U\to V$ consists of positive singular values ${\sigma }_j$ and orthonormal systems $u_j\in U$, $v_j\in V$ satisfyingThey span $\mathcal{N}(K{)}^{\perp }$ and $\overline{\mathcal{R}(K)}$, respectively. The singular value decomposition isFor an infinite-rank compact operator, the singular values tend to zero; for finite rank the sum is finite. Vectors in $\mathcal{N}(K)$ contribute nothing. This convention labels the input singular vectors by $u_j$ and the output ones by $v_j$.

For $(Ku{)}_j=u_j/j$, truncating after $n$ diagonal entries gives finite-rank operators with operator-norm error $1/(n+1)$, proving compactness. Its SVD is

Fix a singular system of a compact operator, with $K{v}_j={\sigma }_j{w}_j$ and $K^{\ast }{w}_j={\sigma }_j{v}_j$, ${\sigma }_j>0$. A spectral regularization method has the formIt annihilates $\mathcal{R}(K{)}^{\perp }$. Standard sufficient spectral filter conditions are boundedness of ${\sup }_{0<\sigma \le \Vert K\Vert }|\sigma g_{\alpha }({\sigma }^2)|$ for each $\alpha$, pointwise $\lambda g_{\alpha }(\lambda )\to 1$ as $\alpha \downarrow 0$ for $\lambda >0$, and a uniform bound on $|\lambda g_{\alpha }(\lambda )|$. These give bounded linear operators and convergence on the domain of the Moore–Penrose inverse of an operator by the Picard criterion and dominated convergence theorem.

For Tikhonov regularization,This can be computed by solving the Tikhonov normal equation, without knowing the singular value decomposition.

For spectral cutoff regularization, takewith value zero at $\lambda =0$. Only singular values ${\sigma }_j\ge \sqrt{\alpha }$ are inverted. Here $g_{\alpha }$ multiplies $\sigma$; conventions calling the full coefficient $\sigma g_{\alpha }({\sigma }^2)$ the spectral filter are equivalent.

Starting from zero, the Landweber iteration isFor $K\ne 0$, take the step size $0<\tau <2/\Vert K{\Vert }^2$. Iterating the linear update givesand its Landweber spectral filter in a singular system of a compact operator isThe strict upper step size bound makes $|1-\tau {\sigma }_j^2|<1$ for every positive singular value. A common more restrictive choice is $0<\tau \le \Vert K{\Vert }^{-2}$, giving nonnegative damping factors. Each finite iterate is bounded and linear; early stopping of Landweber iteration controls the amplification of noise as smaller singular values are progressively inverted. If $K=0$, every iterate is zero for any positive step size.

For Iterated Tikhonov regularization, set $A=K^{\ast }K$ and $B=(I+\tau A{)}^{-1}$. In a singular system of a compact operator, each coefficient obeysSumming this geometric recurrence givesEquivalently, its iterated Tikhonov spectral filter is $g_k(\lambda )=[1-(1+\tau \lambda {)}^{-k}]/\lambda$. For $x\ge 0$, the finite geometric sum givesConsequently the inverse coefficient is at most $\min (k\tau \sigma ,1/\sigma )\le \sqrt{k\tau }$. Thus every $R_{1/k}$ is a bounded linear operator withFor exact admissible data, write $f_j=⟨f,w_j⟩$. The Picard criterion gives ${\sum }_j|f_j{|}^2/{\sigma }_j^2<\infty$, andby dominated convergence theorem. Therefore this is a linear regularization of an inverse problem with discrete parameter $\alpha =1/k$; a family for all positive $\alpha$ can be obtained by setting $k=\max (1,\lceil 1/\alpha \rceil )$. No upper restriction on the positive step size $\tau$ is needed for this implicit iteration.

With exact data $f=K{u}^{†}$, the iteration error obeysThe source condition for quadratic regularization $u^{†}=K^{\ast }v$ excludes any null space component. Using its singular system of a compact operator coefficients,By Bernoulli's inequality, $(1+\tau {\sigma }^2{)}^k\ge 1+k\tau {\sigma }^2$. Also $1+k\tau {\sigma }^2\ge 2\sqrt{k\tau }\sigma$. ThereforeSumming the squared coefficients and using Bessel inequality yieldsThis is an a priori error rate from the source condition for quadratic regularization, with constant independent of $k$.

Write $y={\psi }_s$ and retain the Landweber relaxation parameter $\tau$. The printed $\Vert A\Vert <2$ allows the fixed choice $\tau =1/2$, since then $\tau \Vert A{\Vert }^2<2$. It does not justify a unit step: the scalar linear operator $A=3/2$ has norm below $2$, but unit-step error is multiplied by $1-|A{|}^2=-5/4$ and diverges. We use $0<\tau <2/\Vert A{\Vert }^2$ throughout.

From the recurrence and $f_0=0$, induction yields the closed formFor the compact operator in part ii, take a singular system of a compact operator with $A{v}_j={\sigma }_j{u}_j$, $A^{\ast }{u}_j={\sigma }_j{v}_j$ and ${\sigma }_j>0$. Thus ${\sigma }_j^2$ are the eigenvalues of $A^{\ast }A$; this fixes the notation implicit in the printed hint. In the singular component $j$, the recurrence isSumming the geometric series gives the Landweber spectral filterThere is no contribution from data in $\ker A^{\ast }$ or from $\ker A$ in the reconstruction. Starting at zero is what selects the minimum-norm least-squares solution.

For fixed ${\sigma }_j>0$, $|1-\tau {\sigma }_j^2|<1$, so the numerator tends to $1$. The assumption $y\in D(A^{†})$ is the Picard criterionwith an arbitrary additional component in $\ker A^{\ast }$. The squared reconstruction error isEach term tends to zero and is bounded by a summable term from the Picard criterion. The dominated convergence theorem therefore proves $R_{n}y\to A^{†}y$, where $A^{†}$ is the Moore–Penrose inverse of an operator.

For each finite $n$, the regularization of an inverse problem is stable. Set $t=\tau {\sigma }^2\in (0,2)$. The geometric series and $|1-t|\le 1$ implyConsequently the Landweber noise amplification bound in this general step-size convention is $\Vert R_n\Vert \le \sqrt{2\tau n}$. If additionally $\tau \Vert A{\Vert }^2\le 1$, the sharper numerator bound $\min (nt,1)$ gives $\Vert R_n\Vert \le \sqrt{\tau n}$.

To see the regularization parameter directly, put $\alpha =1/(\tau n)$. Modes with ${\sigma }^2\ll \alpha$ have numerator approximately $n\tau {\sigma }^2$, so their inverse coefficient is approximately $\sigma /\alpha$ rather than $1/\sigma$. Each fixed nonzero mode is eventually restored as $\alpha \downarrow 0$. ThusThe finite iterates suppress unstable small singular values; infinitely many iterations remove this suppression. For noisy data $\Vert y^{\delta }-y\Vert \le \delta$, the noise-bias decomposition for linear regularization givesChoosing $n(\delta )\to \infty$ and ${\delta }^{2}n(\delta )\to 0$ proves noisy-data convergence. The reciprocal iteration index is a regularization parameter, and early stopping controls noise amplification.

Use the singular system of a compact operator convention $A{x}_j={\sigma }_j{y}_j$ and $A^{\ast }{y}_j={\sigma }_j{x}_j$. Starting from zero gives no component in $\ker A$. Write $x(t)={\sum }_j{c}_j(t)x_j$. The coefficient equations areSolving these scalar linear ordinary differential equations givesConsequently asymptotic regularization has the spectral regularization method representationThe component of $f$ orthogonal to the closure of the range of $A$ is annihilated by $A^{\ast }$ and contributes nothing. The scalar filter satisfies $(1-e^{-{\sigma }^2/\alpha })/\sigma \le \min (\sigma /\alpha ,1/\sigma )\le {\alpha }^{-1/2}$, so every fixed-$\alpha$ reconstruction is a bounded linear operator. Its long-time limit is the Moore–Penrose inverse of an operator on admissible data.

For admissible data, the Moore–Penrose inverse of an operator selects the minimum-norm least-squares solution:Its domain is $\mathrm{ran}A\oplus (\mathrm{ran}A{)}^{\perp }$. When the equation is consistent it is the solution of $Ax=y$ orthogonal to the kernel; for inconsistent admissible data it solves the normal equation for a linear inverse problem and projects away the component orthogonal to the range.

Take a singular system of a compact operator satisfying $A{v}_j={\sigma }_j{u}_j$, $A^{\ast }{u}_j={\sigma }_j{v}_j$. The generalized inverse isThe Picard criterion requires the squared coefficients to be summable. For an infinite-rank compact operator, ${\sigma }_j\to 0$, so $\Vert A^{†}{u}_j\Vert =1/{\sigma }_j\to \infty$ even though $\Vert u_j\Vert =1$. Hence $A^{†}$ is unbounded. Finite-rank compact operators do not have this obstruction. Data failing the Picard condition need not have a least-squares minimizer at all, even when the infimum residual is zero.

Let $({\sigma }_i,x_i,y_i)$ be the singular system of a compact operator. Since $P_n{x}_i=x_i$ for $i\le n$ and $P_n{x}_i=0$ for $i>n$, the truncated operator $T_n=A{P}_n$ has the finite singular value decompositionConsequentlyso $\boxed{T_n^{†}=A^{†}{Q}_n}$.

For completeness, $T_n^{†}{T}_n=P_n$ on $(\ker A{)}^{\perp }$ and $T_n{T}_n^{†}=Q_n$. These are self-adjoint orthogonal projectors, and henceThus all four Penrose equations hold, which verifies the formula independently of the singular expansion.