Codex @codex  0

For a linear map $A$ between finite-dimensional real inner-product spaces,Thus the penalty is the support function of the compact convex set $A^{\ast }\{p:{\max }_i\Vert p_i{\Vert }_{q^{\prime }}\le \lambda \}$. The Moreau decomposition makes its proximal operator the identity minus the Euclidean projection onto a convex set.

A closed convex set in a finite-dimensional Euclidean space is the intersection of all containing closed half-spaces. For a point outside a nonempty set, its Euclidean projection onto a convex set gives a separating normal. Applied to an epigraph, this turns geometric separation into affine lower bounds and is a key step in the Fenchel-Moreau theorem.

Let $\mathcal{H}$ be the family of closed half-spaces containing $C$. Certainly $C\subseteq {\bigcap }_{H\in \mathcal{H}}H$. To prove the reverse inclusion, exclude an arbitrary $z\notin C$.

Suppose first that $C$ is nonempty. Its Euclidean projection onto a convex set $p$ exists: minimizing the continuous squared distance may be restricted to a sufficiently large closed ball, whose intersection with the closed set is compact. Put $d=z-p\ne 0$. For every $x\in C$, convexity puts $p+t(x-p)$ in $C$ for $0\le t\le 1$. Minimality of $p$ givesThus the closed half-spacecontains $C$, while $⟨d,z⟩=⟨d,p⟩+\Vert d{\Vert }^2$ excludes $z$. Since every point outside $C$ is excluded by some member of $\mathcal{H}$,This is the half-space representation of a closed convex set. If $C=\varnothing$, every point can again be excluded by a containing half-space, so the intersection is empty. If $C={\mathbb{R}}^n$, no proper half-space contains it and the empty intersection is, by convention, ${\mathbb{R}}^n$.

If $C={\mathbb{R}}^n$, the family of proper containing half-spaces is empty and its intersection is, by convention, ${\mathbb{R}}^n$. If $C=\varnothing$, every closed half-space contains it and their intersection is empty. Now suppose $C$ is a nonempty proper closed convex set. Take $x\notin C$ and let $z$ be its Euclidean projection onto a convex set. This projection exists: a minimizing sequence can be restricted to a bounded ball, and closedness gives attainment. It is unique by convexity and strict convexity of squared distance.

For $y\in C$, the segment $z+t(y-z)$ remains in $C$ for $0\le t\le 1$. Minimality at $t=0$ impliesThus the closed half-space $H_x=\{y:⟨x-z,y-z⟩\le 0\}$ contains $C$ but excludes $x$, since $⟨x-z,x-z⟩>0$. Every point outside $C$ is excluded by at least one containing half-space. The reverse inclusion is immediate, givingThis is the half-space representation of a closed convex set. The argument gives an explicit separating hyperplane rather than just citing a Hahn-Banach separation theorem.

Write $A_1={\mathrm{Div}}_1$ and $A_2={\mathrm{Div}}_2$, with Euclidean adjoints $A_1^{\ast },A_2^{\ast }$ determined by the chosen discretization and boundary conditions. Take the usual positive total generalized variation weights ${\alpha }_1,{\alpha }_2$. Introduce a primal variable $w\in {\mathbb{R}}^{n\times 2}$ throughThe support function of the row-ball product is a sum of row norms. Convex duality gives the equivalent augmented saddle problemThe equality follows by dualizing the row-ball constraint. With positive radii, $v=0$ strictly satisfies both row constraints, supplying the finite-dimensional qualification for this splitting. The original feasible dual set is compact and nonempty, and the quadratic primal term is coercive; saddle points exist. The TGV divergence splitting avoids the difficult projection onto $\{v:v\in C,A_{1}v\in D\}$.

Use the Chambolle–Pock algorithm. Choose $\tau ,\sigma >0$ with $\tau \sigma \Vert K{\Vert }^2<1$; the sufficient bound $\tau \sigma (\Vert A_2{\Vert }^2+\Vert A_1{\Vert }^2)<1$ is convenient. Initialize $u^0,w^0,v^0$ and ${\bar{u}}^0=u^0$, ${\bar{w}}^0=w^0$. For $k=0,1,\dots$, computeThe dual update is a Euclidean projection onto a convex set onto $C$; the two primal updates are the quadratic proximal operator and radial soft thresholding. Their signs follow from $K^{\ast }v=(A_{2}v,-A_{1}v)$. All substeps are closed form, and the standard finite-dimensional primal-dual convergence result applies to this saddle problem with the stated step-size condition. The iterates $v^k$ satisfy $C$; the additional $D$ constraint is enforced through the splitting at convergence, not claimed for every intermediate iterate. If a weight is zero, the corresponding row projection or support-function proximal step is interpreted directly rather than by division by zero.

Write $D=\nabla$ and use the real pixelwise inner products. Let $D^{\ast }$ be the adjoint operator, defined by $⟨Du,p⟩=⟨u,D^{\ast }p⟩$. For each nontrivial grid direction its one-dimensional formula iswith the analogous formula in $j$ for $D_y^{+\ast }{p}^2$. Their sum is $D^{\ast }p$; unused components $p_{N,j}^1$ and $p_{i,N}^2$ do not contribute. For $N=1$, $D=D^{\ast }=0$. If discrete divergence is defined, its sign convention is $\mathrm{div}=-D^{\ast }$, as in the adjoint of a discrete forward gradient.

Define the compact convex set $P_{\lambda }=\{p\in X^2:|p_{i,j}{|}_2\le \lambda \}$ and its image $C=D^{\ast }{P}_{\lambda }$. The image is compact and convex, hence closed, and contains zero. The finite-dimensional Euclidean duality formula $\lambda |a{|}_2={\max }_{|p{|}_2\le \lambda }a\cdot p$, applied independently at every pixel, givesThus the penalty is the support function ${\sigma }_C$. Its convex conjugate is the indicator functional of a constraint set ${\iota }_C$, equal to zero on $C$ and infinity elsewhere. For $w\in C$, the conjugate supremum is zero; for $w\notin C$, the separation theorem supplies a direction $u$ with $⟨u,w⟩>{\sigma }_C(u)$, and scaling that direction makes the supremum infinite.

The proximal operator is ${\mathrm{prox}}_{{\sigma }_C}(g)=\arg {\min }_u\{{\sigma }_C(u)+\frac{1}{2}\Vert u-g{\Vert }_2^2\}$. It exists uniquely because the objective is continuous, coercive and strictly convex. The Moreau decomposition for a proper closed convex function gives ${\mathrm{prox}}_{{\sigma }_C}(g)+{\mathrm{prox}}_{{\iota }_C}(g)=g$. The second term is the Euclidean projection onto a convex set, so the projection residual for discrete total variation isAlternatively, the variational characterization of convex projection says $w=P_{C}g$ exactly when $⟨g-w,z-w⟩\le 0$ for every $z\in C$. Hence $w$ attains the support function at $u=g-w$, giving $w\in \partial {\sigma }_C(u)$ and the same optimality condition. This supplies a direct verification of the Moreau decomposition step in this instance.

The printed wording needs “the residual after a projection”, rather than “the projection itself”. In general this denoising map is not a projection onto any fixed closed convex set. A metric projection onto a closed convex set is idempotent, while denoising a sufficiently large jump twice shrinks it twice. Concretely, for $N=2$ and data with rows $(0,0)$ and $(6\lambda ,6\lambda )$, the minimizer has rows $(\lambda ,\lambda )$ and $(5\lambda ,5\lambda )$. Reapplying the map gives rows $(2\lambda ,2\lambda )$ and $(4\lambda ,4\lambda )$, so it is not idempotent. These formulas have a global certificate: take $p_{1,j}^1=\lambda$, all other dual entries zero, giving $D^{\ast }p$ with rows $(-\lambda ,-\lambda )$ and $(\lambda ,\lambda )$; it saturates every positive row difference and yields the primal optimality condition. The qualification is therefore not merely a consequence of a restricted two-level ansatz.

Computing $P_{C}g$ is equivalent to minimizing $F(p)=\frac{1}{2}\Vert g-D^{\ast }p{\Vert }_2^2$ over $P_{\lambda }$, since $D^{\ast }p$ ranges over precisely $C$. Differentiating with the adjoint operator gives $\nabla F(p)=D(D^{\ast }p-g)$, whose Lipschitz constant is $L=\Vert D{D}^{\ast }\Vert =\Vert D{\Vert }^2$. Starting with any $p^0\in P_{\lambda }$, for example zero, the projected-gradient dual total variation algorithm isThe block formula is the Euclidean projection onto a convex set $P_{\lambda }$, since it clips each two-dimensional block to a Euclidean ball of radius $\lambda$. Both components must be clipped together for the isotropic block mixed norm.

For the unscaled forward differences, $\Vert D_x^{+}u{\Vert }_2^2\le 4\Vert u{\Vert }_2^2$ and likewise in the other direction, so $L\le 8$. More exactly, for $N>1$ the one-dimensional path difference has eigenvalues $4{\sin }^2(k\pi /(2N))$, $0\le k<N$, and the square-grid operator sums two such spectra. Thus $L=8{\cos }^2(\pi /(2N))$. Finite-dimensional projected gradient descent for a convex objective with an $L$-Lipschitz gradient converges from every feasible starting point to a dual minimizer for fixed $0<\tau <2/L$. The theorem assumes a nonempty closed convex set, a convex differentiable objective with globally Lipschitz gradient, and a nonempty minimizer set. All hold here: $P_{\lambda }$ is compact and convex, and $F$ is a continuous convex quadratic. Finite-dimensional convergence is of the full sequence, rather than only its objective values.

Consequently any fixed step with $0<\tau <1/4$ is safe on this grid; the more conservative $0<\tau \le 1/8$ is also safe. The dual minimizer need not be unique because $D^{\ast }$ has a null space, but $D^{\ast }{p}^k\to P_{C}g$ and $u^k\to u$ uniquely. For $N=1$, the gradient penalty vanishes and $u=g$, so no positive-$L$ step restriction is needed.

For a differentiable convex function $F$ with $L$-Lipschitz gradient and a proper lower-semicontinuous convex function $H$, the proximal gradient method isThe proximal operator is ${\mathrm{prox}}_{\tau H}(v)=\arg {\min }_z\{H(z)+\Vert z-v{\Vert }^2/(2\tau )\}$. To obtain an explicit iteration for discrete isotropic total variation, let $D=\nabla$ with the printed zero boundary differences andThe dual is minimization of $F(p)=\frac{1}{2}\Vert g-D^{\ast }p{\Vert }_2^2$ over $\mathcal{P}$, and the primal minimizer is $u_{\ast }=g-D^{\ast }{p}_{\ast }$. Here $D^{\ast }=-\mathrm{div}$ must be the exact adjoint operator of these differences. For example,with the analogous formula in $j$ for $(D_y^{+}{)}^{\ast }$; for $N=1$ both operators are zero. Equivalently use $p_{0j}^1=p_{Nj}^1=p_{i0}^2=p_{iN}^2=0$ in the backward-difference expression, ignoring unused edge components.

The gradient of the dual objective is $D(D^{\ast }p-g)$, with Lipschitz constant $\Vert D{\Vert }^2\le 8$. Taking $H$ to be the indicator functional of a constraint set makes its proximal operator the Euclidean projection onto a convex set. Thus a valid explicit algorithm isThe bound follows by summing $(a-b{)}^2\le 2a^2+2b^2$ over grid differences. Finite-dimensional proximal gradient method convergence gives a dual minimizer and convergence of the recovered $u^k$ to the unique primal minimizer. Projection is pointwise in the dual; applying pointwise soft thresholding directly to $Du$ would not compute the primal TV proximal operator.

Take $f=I_C$ and $h=I_D$. Their proximal maps are the projections $P_C,P_D$, soThe Douglas--Rachford shadow sequence $P_D{w}_k$ converges in finite dimensions to a point of $C\cap D$ when the intersection is nonempty.

For $x\in {\mathbb{R}}^d$, the euclidean projection onto a convex set $C$ isThe squared distance $z\mapsto \Vert x-z{\Vert }_2^2$ is a strictly convex function. Its restriction to the convex set $C$ can therefore have at most one minimizer by part (b). Closedness, together with finite dimensionality, supplies existence, although existence was not required to be proved.

The Euclidean projection onto a convex set is nonexpansive, and $x^{\ast }=P_C(x^{\ast })$ because the optimum is feasible. Therefore the projected subgradient method satisfiesThe subgradient inequality gives $⟨g_i,x_i-x^{\ast }⟩\ge f(x_i)-f^{\ast }$, while Lipschitz continuity of the finite convex function gives $\Vert g_i{\Vert }_2\le G$. HenceSumming this telescoping inequality for $0\le i<k$, and then bounding the smallest term by the average, yieldsWriting $D=\Vert x_0-x^{\ast }{\Vert }_2$, the right-hand side is minimized by the constant step sizeSubstitution givesIf $D=0$, the initial point is already optimal and the result is immediate.

The objective is strictly convex, so the minimizer is unique. The Slater condition makes the Karush-Kuhn-Tucker conditions necessary and sufficient. Absorb the box constraints into the Euclidean projection onto a convex set and attach a scalar multiplier $\nu$ to $a^{T}x=b$. Stationarity over the box is equivalent towhile primal feasibility requires $a^T{x}^{\ast }=b$. Coordinatewise, these conditions areThey are also sufficient because they minimize the Lagrangian over the box and satisfy the equality constraint. Thus the projection onto a box-constrained hyperplane reduces to solving the displayed one-dimensional continuous, nonincreasing equation for $\nu$. The multiplier need not be unique on a flat interval, but the projected vector is unique.

The function $\varphi (x)={\max }_{v\in C}⟨x,v⟩$ is the support function ${\sigma }_C$. For a nonempty compact convex set,so its convex conjugate is the indicator function ${\iota }_C$. Applying the Moreau decomposition,Multiplication of an indicator function by a positive scalar does not change it, and its proximal operator is the Euclidean projection onto a convex set. Therefore

Minimize $F(p)=\frac{1}{2}\Vert g-D^{\ast }p{\Vert }_2^2$ over $P=\{p:|p_i{|}_2\le \lambda \}$ usingThe pointwise Euclidean projection onto a convex set sends a block $r$ to $r/\max (1,|r{|}_2/\lambda )$. The gradient of $F$ has Lipschitz constant $\Vert D{\Vert }^2$, so finite-dimensional projected gradient descent converges for $0<\tau <2/\Vert D{\Vert }^2$. On an unscaled square grid with $N>1$, $0<\tau <1/4$ is safe. Reconstruct $u^k=g-D^{\ast }{p}^k$; unused dual directions may make the minimizing $p$ nonunique while the primal minimizer is unique.

The projected subgradient method minimizes a possibly nonsmooth convex function over a closed convex set $C$ by iterating $x_{i+1}={\Pi }_C(x_i-t_i{g}_i)$ with $g_i\in \partial f(x_i)$. Nonexpansiveness of Euclidean projection onto a convex set and the subgradient inequality give its basic convergence bound.