Past exam of the mathematics course of the University of Cambridge / 2022 / iii / Paper 326 / 2

The extended-real functional $f:X\to \overline{\mathbb{R}}$ is sequentially lower semicontinuous in the topology ${\tau }_X$ when every sequence $x_n\to x$ in ${\tau }_X$ satisfies

The assertion is understood for every $\eta \in \mathbb{R}$ and for a topology in which sequentially closed sets are closed, in particular the norm topology of a Banach space. Suppose first that $f$ is sequentially lower semicontinuous and $x_n\in {\Omega }_{\eta }=\{x:f(x)\le \eta \}$ converges to $x$. Thenso $x\in {\Omega }_{\eta }$ and the sublevel set is closed. Conversely, if all sublevel sets are closed but lower semicontinuity fails, there are $x_n\to x$ and a real $\eta <f(x)$ with a subsequence satisfying $f(x_{n_k})\le \eta$. Closedness of ${\Omega }_{\eta }$ would put $x$ in that set, a contradiction. This proves the closed-sublevel-set characterization of lower semicontinuity.

Use the extended-real characteristic functionalFor $\eta <0$ its sublevel set is empty, while for every finite $\eta \ge 0$ its sublevel set is $X^{\prime }$. Both are closed when $X^{\prime }$ is closed, so part ii proves that this indicator functional of a constraint set is lower semicontinuous.

Let $T_{\alpha }=A^{\ast }A+\alpha I$. For every $f\in X$,Thus $T_{\alpha }$ is a coercive operator. If it is invertible and $T_{\alpha }f=g$, the lower bound and the Cauchy-Schwarz inequality giveand therefore $\Vert T_{\alpha }^{-1}g\Vert \le {\alpha }^{-1}\Vert g\Vert$. Hence

Subtracting $T_{\alpha }{u}_n=z_n$ and $T_{\alpha }{u}_m=z_m$ and applying the coercive estimate from part i yieldsThus $(z_n)$ Cauchy implies $(u_n)$ Cauchy. Completeness of the Hilbert space $X$ gives $u_n\to u\in X$, and boundedness of $T_{\alpha }$ gives $z_n=T_{\alpha }{u}_n\to T_{\alpha }u$.

The lower bound in part i shows that $T_{\alpha }u=0$ implies $u=0$, so $T_{\alpha }$ is injective. Since $T_{\alpha }$ is self-adjoint,so its range is dense. Part ii shows that its range is also closed: a convergent sequence $z_n=T_{\alpha }{u}_n$ has Cauchy preimages, whose limit $u$ satisfies $T_{\alpha }u=\lim z_n$. Hence $\mathcal{R}(T_{\alpha })=X$. The operator is bijective, and the estimate in part i proves that its inverse is bounded. Therefore $A^{\ast }A+\alpha I$ is invertible for every $\alpha >0$.

The equation $(A^{\ast }A+{\alpha }_{\ast }I)u_{\ast }=A^{\ast }f$ is the Tikhonov normal equation. For $u=u_{\ast }+h$, expand the Tikhonov regularization functional:The normal equation makes the linear term zero. The final two terms are nonnegative and are strictly positive for $h\ne 0$ because ${\alpha }_{\ast }>0$. Thus $u_{\ast }$ is the unique global minimizer.

The objective separates by coordinates, so minimizeThe subgradient optimality condition isFor $z>0$ this gives $z=x_i-\lambda$, valid when $x_i>\lambda$; for $z<0$ it gives $z=x_i+\lambda$, valid when $x_i<-\lambda$. At $z=0$, the condition is $x_i\in [-\lambda ,\lambda ]$. Therefore the shrinkage operator is the soft-thresholding operator

The graph is continuous and piecewise linear: it is horizontal at zero on $[-\lambda ,\lambda ]$ and has slope one outside that interval, joining the axis at $(-\lambda ,0)$ and $(\lambda ,0)$.

 psi
  ^
  |                    /
  |                   /
--+---------==========--------> x
 /         -lambda  lambda
/