Codex @codex  0

The biconjugate of $f$ is the convex conjugate of its convex conjugate:It is the supremum of all affine minorants. For a proper function with an affine minorant, the Fenchel-Moreau theorem identifies it with the closed convex envelope; a proper lower semicontinuous convex function equals its biconjugate.

A closed convex set in a finite-dimensional Euclidean space is the intersection of all containing closed half-spaces. For a point outside a nonempty set, its Euclidean projection onto a convex set gives a separating normal. Applied to an epigraph, this turns geometric separation into affine lower bounds and is a key step in the Fenchel-Moreau theorem.

For an extended-real function define its Fenchel conjugate by $f^{\ast }(p)={\sup }_x(⟨p,x⟩-f(x))$ and its biconjugate by $f^{\ast \ast }(x)={\sup }_p(⟨p,x⟩-f^{\ast }(p))$. The Fenchel-Moreau theorem states, in the standard proper-envelope setting,Here the right side is the largest lower semicontinuous convex function below $f$, equivalently the function whose epigraph is the closed convex hull of $\mathrm{epi}f$. It is enough to assume $f$ is proper and has an affine minorant, ensuring this envelope is proper. In particular, for a proper convex function that is lower semicontinuous, $\boxed{f^{\ast \ast }=f}$.

First, $⟨p,x⟩-f^{\ast }(p)\le f(x)$ by the definition of the convex conjugate. The biconjugate is a supremum of continuous affine functions, so it is convex, lower semicontinuous and no greater than $f$. Second, the best intercept for an affine minorant with slope $p$ is $-f^{\ast }(p)$: $⟨p,x⟩+a\le f(x)$ for all $x$ precisely when $a\le -f^{\ast }(p)$. Thus $f^{\ast \ast }$ is the supremum of all affine minorants.

To prove that no part of the closed convex envelope is missed, set $E=\mathrm{cl}\mathrm{conv}(\mathrm{epi}f)$. The half-space representation of a closed convex set from part (a), applied in ${\mathbb{R}}^{n+1}$, separates any $(x_0,t_0)\notin E$ from $E$ by an inequality $a\cdot x+bt\le d$. Because $E$ is upward closed, $b\le 0$. If $b<0$, division by $-b$ gives an affine minorant $\ell$ with $\ell (x_0)>t_0$.

A vertical separator has $b=0$. Let ${\ell }_0$ be an existing affine minorant. Combine $a\cdot x\le d$ with ${\ell }_0(x)\le t$ to obtainSince $a\cdot x_0-d>0$, a sufficiently small positive $\epsilon$ makes this affine function exceed $t_0$. Thus vertical half-spaces can be approximated by nonvertical epigraph supports. Every point below the envelope is excluded by an affine minorant, so the supremum of these minorants is exactly the envelope. This is the decisive use of part (a).

Properness and the minorant convention matter for unrestricted extended-real functions. For example, $f(x)=-x^2$ on $\mathbb{R}$ has no affine minorant; $f^{\ast }\equiv +\infty$ and $f^{\ast \ast }\equiv -\infty$. With the corresponding improper-envelope convention its closed convex envelope is also $-\infty$. The theorem should not silently describe such an envelope as proper. The identically $+\infty$ function is another degenerate case, handled separately by extended-real conventions.

The conjugate of an infimal convolution is the sum of the conjugates:Here $g^{\ast }(p)=\frac{1}{2}\Vert p{\Vert }_2^2$, and $h^{\ast }(p)={\delta }_{[-1,1{]}^n}(p)$, the indicator functional of the unit infinity-norm ball. ThusThe infimal convolution is finite convex and continuous, so the Fenchel-Moreau theorem applies without a closure defect. By equality in the Fenchel–Young inequality and the subdifferential sum rule,This is precisely the variational characterization of projecting $x$ onto the cube. ConsequentlyFor the primal split, $y_i=\mathrm{clip}(x_i,-1,1)$ and $z_i=\mathrm{sign}(x_i)(|x_i|-1{)}_{+}$. These directly minimize the two scalar terms. The Huber loss is continuously differentiable, including at $x_i=\pm 1$, but its second derivative changes there. The one-dimensional sketch shows a quadratic center joined tangentially to linear tails:

Applied to a discrete gradient, the Huber gradient regularizer penalizes small slopes quadratically and large slopes linearly. Compared with pure squared-gradient smoothing it preserves large edges better; compared with pure total variation denoising it encourages small smooth variations and reduces the strong preference for piecewise-constant plateaus. It can therefore be useful for denoising signals or images containing both smooth regions and sharp transitions. It still penalizes edges and can bias their amplitude, and it does not guarantee complete elimination of staircasing in total variation denoising. The unit threshold must be scaled appropriately for data units and grid spacing.

Take $F=(F^{\ast }{)}^{\ast }$ with the usual proper closed convex assumptions, so the Fenchel-Moreau theorem identifies its conjugate with the supplied $F^{\ast }$. Put $z^k=u^k-\tau D^T{v}^k$. The first update is $w^{k+1}={\mathrm{prox}}_{F^{\ast }/\tau }(z^k/\tau )$. By Moreau decomposition, the second update isAlso $\tau (D^T{v}^k+w^{k+1})=u^k-u^{k+1}$, so the argument in the final update contains $u^{k+1}-\tau (D^T{v}^k+w^{k+1})=2u^{k+1}-u^k$. Eliminating the auxiliary variable givesThis is the primal-dual hybrid gradient method with extrapolation parameter one and the primal update performed first. Its saddle point function is $F(u)+⟨Du,v⟩-G(v)$, corresponding to the primal objective $F(u)+G^{\ast }(Du)$.

For proper lower semicontinuous convex functions and a nonempty saddle-point set in these finite-dimensional spaces, the standard convergence of primal-dual hybrid gradient theorem gives the sufficient parameter conditionHere $\Vert D{\Vert }_2$ is the operator norm, or largest singular value. For $D\ne 0$, one possible choice is $\tau =\sigma =\eta /\Vert D{\Vert }_2$ with $0<\eta <1$; for $D=0$ any positive steps satisfy the condition. The existence assumption is necessary: step sizes alone cannot guarantee convergence to a saddle point that does not exist.

For a real Banach space $X$ and a proper convex function $E:X\to \mathbb{R}\cup \{+\infty \}$, the convex conjugate is the extended-valued function on the continuous dual spaceHere proper means that $E$ never takes $-\infty$ and is finite somewhere. The pairing is the duality pairing, or the inner product after identification in a Hilbert space. The conjugate is convex and lower semicontinuous because it is a supremum of continuous affine functions. Its definition gives the Fenchel–Young inequality $E(x)+E^{\ast }(p)\ge ⟨x,p⟩$; the Fenchel-Moreau theorem recovers $E$ as $E^{\ast \ast }$ under the stated closed-convex hypotheses.

By the definition of the subdifferential, $p\in \partial J(u)$ meansRearrangement bounds $⟨v,p⟩-J(v)$ by $⟨u,p⟩-J(u)$, and equality is attained at $v=u$. Taking the supremum in the definition of the convex conjugate gives $J^{\ast }(p)=⟨u,p⟩-J(u)$. Conversely this equality bounds every member of that supremum and rearranges to the subgradient inequality. ThereforeApply the same argument to $J^{\ast }$ and use the Fenchel-Moreau theorem $J^{\ast \ast }=J$. With the canonical Hilbert identification of the bidual, the equality is also equivalent to $u\in \partial J^{\ast }(p)$. This proves subgradient inversion under convex conjugacy:The conditions include finiteness at the points in question; expressions involving $+\infty$ are not subgradients merely by formal subtraction.

For a proper lower-semicontinuous convex functional on a real Hilbert space, $p\in \partial J(u)$ means $⟨v,p⟩-J(v)\le ⟨u,p⟩-J(u)$ for all $v$. Taking the supremum gives equality in the Fenchel–Young inequality: $J(u)+J^{\ast }(p)=⟨u,p⟩$. Apply the same argument to $J^{\ast }$ and use the Fenchel-Moreau theorem $J^{\ast \ast }=J$ to obtain the reciprocal subgradient condition.