Residual method for variational regularization

Given $\Vert f_{\delta }-f\Vert \le \delta$, the residual method minimizes a penalty $J(u)$ subject to $\Vert Au-f_{\delta }\Vert \le \delta$. Every exact solution is feasible, so a reconstruction $u_{\delta }$ has penalty no larger than that of any J-minimizing solution.

Suppose $J$ is nonnegative and sequentially lower semicontinuous, and its sublevel sets are strongly sequentially compact. For a solvable exact equation $Au=f$, every sequence of residual-method minimizers with $\delta \to 0$ has a strongly convergent subsequence, and every limit is a J-minimizing solution. Indeed, comparison with an exact penalty minimizer bounds all reconstructions in one compact sublevel set; the residual bound gives $\Vert A{u}_{\delta }-f\Vert \le 2\delta$. Continuity of $A$ and lower semicontinuity of $J$ identify each subsequential limit as an exact minimizer. Thus the distance to a set of exact penalty minimizers tends to zero. Uniqueness of the exact minimizer upgrades this to full strong convergence.

Uniqueness cannot be omitted: for $A=0$, $f_{\delta }=f=0$, and $J(u)=\max \{|u|-1,0\}$ on $\mathbb{R}$, the sublevel sets are compact intervals but the minimizers $u_{1/k}=(-1{)}^k$ do not converge.