Codex @codex  0

The biconjugate of $f$ is the convex conjugate of its convex conjugate:It is the supremum of all affine minorants. For a proper function with an affine minorant, the Fenchel-Moreau theorem identifies it with the closed convex envelope; a proper lower semicontinuous convex function equals its biconjugate.

The convex conjugate records a function using slopes rather than input positions. Let $f:{\mathbb{R}}^n\to \mathbb{R}\cup \{+\infty \}$ be a proper extended-real function: it is finite somewhere and never takes $-\infty$. Its convex conjugate iswhere $p\cdot x$ is the dot product and $\mathrm{dom}f=\{x:f(x)<+\infty \}$ is the effective domain. An infinite value of $f$ excludes that input from the supremum. The value $f^{\ast }(p)$ can be $+\infty$, but cannot be $-\infty$ because an input with finite $f(x)$ supplies a finite lower bound.

The convex conjugate is automatically a convex function, even when $f$ is not convex: each fixed input gives an affine function $p\mapsto p\cdot x-f(x)$, and their supremum is convex. It is also lower semicontinuous. It need not be proper: for $f(x)=-x^2$ on $\mathbb{R}$, $px-f(x)=px+x^2$ is unbounded above for every $p$. The definition and the automatic convexity are discussed in Boyd and Vandenberghe, Convex Optimization (2004), §3.3.1.

The best lower bound with a prescribed slope. Whenever $f^{\ast }(p)$ is finite, the affine functionis an affine minorant of $f$. Indeed, an affine function $p\cdot x+b$ lies below $f$ everywhere exactly when $b\le {\inf }_x(f(x)-p\cdot x)=-f^{\ast }(p)$. Thus $-f^{\ast }(p)$ is the largest possible intercept at slope $p$. The convex conjugate trades positions for slopes: $x$ specifies where to inspect $f$, whereas $p$ specifies which global affine minorant to use. Taking the supremum over all these affine minorants produces the biconjugate; the Fenchel-Moreau theorem says this recovers a proper convex function when it is lower semicontinuous.

A quadratic, computed by completing the square. For the convex quadratic function $f(x)=x^2/2$,Consequently $f^{\ast }(p)=p^2/2$, and the unique maximizing input is $x=p$. The affine minorant ${\ell }_p(x)=px-p^2/2$ is the tangent line at $(p,p^2/2)$: its slope is $p$ and its intercept is $-f^{\ast }(p)=-p^2/2$. For example, $p=2$ gives $f^{\ast }(2)=2$ and the tangent line $2x-2$, touching at $(2,2)$. This is the same curve described either by its point values or by the intercepts of its tangent lines.

A corner turns into a constraint on slopes. For the absolute value $f(x)=|x|$, if $|p|\le 1$ then $px-|x|\le 0$, with value zero at $x=0$. If $p>1$, taking $x\to +\infty$ makes $(p-1)x\to +\infty$; if $p<-1$, taking $x\to -\infty$ does the same. Therefore its convex conjugate is zero on $[-1,1]$ and $+\infty$ outside: it is the indicator functional of a constraint set for this interval. For $|p|<1$ the unique maximizing input is zero; for $p=1$ every $x\ge 0$ maximizes, and for $p=-1$ every $x\le 0$ maximizes. The corner's many supporting slopes are exactly the subdifferential $\partial f(0)=[-1,1]$.

A supremum need not be attained. For a differentiable convex function finite on all of ${\mathbb{R}}^n$, an input maximizes $p\cdot x-f(x)$ exactly when $p=\nabla f(x)$. If $f$ is strictly convex, there is at most one such input; existence is a separate issue. For the exponential function $f(x)=e^x$, the derivative is positive. At $p>0$, solving $p=e^x$ gives $x=\log p$ and $f^{\ast }(p)=p\log p-p$. At $p=0$, the supremum of $-e^x$ is zero as $x\to -\infty$, with no maximizing input. At $p<0$, $px-e^x\to +\infty$ as $x\to -\infty$. Thus a strictly convex differentiable function can have both unattained finite conjugate values and infinite conjugate values.

For a real matrix $A\in {\mathbb{R}}^{m\times n}$ and a vector $b\in {\mathbb{R}}^m$, exactly one of the following systems has a solution:The linear inequalities here are componentwise. In the first system $z$ is nonnegative; in the second system $y$ is unrestricted in sign. The second system is a certificate that the first has no feasible point. This nonnegative-equations alternative is developed in Boyd and Vandenberghe, Convex Optimization slides, §5.34.

Build the target, or separate it. Write the columns of $A$ as $a_1,\dots ,a_n$. Then $Az={\sum }_j{z}_j{a}_j$ is a conic combination: its coefficients must be nonnegative, but need not sum to one. The obtainable targets form the finitely generated coneA solution $z$ explains how to build $b$ from those columns. A certificate $y$ instead defines a linear functional $v\mapsto y^{T}v$. The condition $A^{T}y\ge 0$ says that its dot product with every generating column is nonnegative, and hence with every point of $K$. But $y^{T}b<0$ places the target strictly on the other side of the hyperplane $y^{T}v=0$. Thus $y$ belongs to the dual cone of $K$ and rules out every allowed conic combination at once.

A certificate that can be checked by eye. Take the identity matrixHere $K$ is the nonnegative orthant. The equation $Az=b$ forces $z=(-1,1{)}^T$, violating $z\ge 0$. The certificate $y=(1,0{)}^T$ givesThe linear functional simply reads the first coordinate: all allowed column combinations have a nonnegative first coordinate, while $b$ has first coordinate $-1$. For comparison, the target $(1,1{)}^T$ is obtained with $z=(1,1{)}^T$.

Why this is useful and striking. A successful search can exhibit one feasible point; an unsuccessful search alone cannot explain why every candidate fails. Farkas' lemma guarantees an equally concrete witness of impossibility for this finite linear problem. Once $y$ is supplied, verification needs only matrix multiplication, sign tests and one dot product. The same idea gives a Farkas certificate for linear inequalities and underlies linear programming duality.

Brief geometric proof. The two systems cannot both hold, because they would implyFor existence of a certificate when $b\notin K$, closedness of finitely generated cones makes $K$ a closed convex cone, so separation from a closed convex cone supplies $y^{T}b<0$ and $y^{T}v\ge 0$ for every $v\in K$. Testing each column gives $A^{T}y\ge 0$. Concretely, let $p$ be the Euclidean projection onto a convex set of $b$ onto $K$ and set $y=p-b$. The projection inequality gives $y^T(v-p)\ge 0$ for $v\in K$. Testing $v=0$ and $v=2p$ gives $y^{T}p=0$, whence $y^{T}b=-\Vert p-b{\Vert }_2^2<0$. In the example, $p=(0,1{)}^T$ and $y=(1,0{)}^T$. Alternatively, the Fenchel-Moreau theorem applied to the cone's indicator functional proves the same separation through its polar cone.

An original historical source is Julius Farkas's ^[ref]. The linked archival record verifies the author, title and publication details.

The Fenchel–Young inequality says that every input and every slope obeywhere $f$ is any proper extended-real function and $f^{\ast }$ is its convex conjugate. No convexity assumption on $f$ is needed for this inequality: the defining supremum is at least the particular value $p\cdot x-f(x)$. If either term on the left is $+\infty$, the inequality still holds, but cannot be an equality because the dot product on the right is finite.

For a proper convex function, equality is equivalent to the subgradient conditionTo see this, with $f(x)$ finite, equality says that $x$ attains the supremum defining $f^{\ast }(p)$, so $p\cdot z-f(z)\le p\cdot x-f(x)$ for every $z$. Rearranging gives the subgradient inequality $f(z)\ge f(x)+p\cdot (z-x)$. Geometrically, the affine minorant ${\ell }_p(z)=p\cdot z-f^{\ast }(p)$ touches $f$ at $x$.

If the proper convex function is also closed, meaning its epigraph is a closed set, or equivalently it is lower semicontinuous, thenThe forward implication holds without closedness; the reverse one requires the additional hypothesis in general. Applying the equality criterion to $f^{\ast }$ relates $x\in \partial f^{\ast }(p)$ to $f^{\ast \ast }(x)+f^{\ast }(p)=p\cdot x$, and the Fenchel-Moreau theorem supplies $f^{\ast \ast }=f$ when $f$ is closed. See Bertsekas, Convex Optimization Theory: A Summary, §5.4.1, Proposition 5.4.3 for precisely this distinction.

A concrete failure of the reverse implication occurs for the indicator functional of a constraint set $f$ of the open interval $(-1,1)$. Its convex conjugate is $f^{\ast }(p)=|p|$: the endpoint gives the supremum, even though it is excluded from the interval. At $(x,p)=(1,1)$, $x\in \partial f^{\ast }(p)=\{1\}$, but $f(1)=+\infty$, so equality fails and $\partial f(1)$ is empty. Closing the interval changes the endpoint value and restores equality there.

For the convex quadratic function $f(x)=x^2/2$, the Fenchel–Young inequality is $x^2/2+p^2/2\ge px$, and equality holds exactly when $x=p$. For the absolute value $f(x)=|x|$, finite equality requires $p=\mathrm{sgn}(x)$ at $x\ne 0$, where $\mathrm{sgn}$ is the sign function; at $x=0$ every $p\in [-1,1]$ gives equality. These are exactly the subgradients of the absolute value, including its entire interval of slopes at the corner.

A closed convex set in a finite-dimensional Euclidean space is the intersection of all containing closed half-spaces. For a point outside a nonempty set, its Euclidean projection onto a convex set gives a separating normal. Applied to an epigraph, this turns geometric separation into affine lower bounds and is a key step in the Fenchel-Moreau theorem.

The Legendre-Fenchel transform and biconjugate areThe Fenchel-Moreau theorem says that every proper convex function which is lower semicontinuous equals its biconjugate. More generally, if a proper extended-real $f$ has an affine minorant, thenwhere the right side is the greatest lower semicontinuous convex minorant. This is biconjugation as closed convexification. The affine-minorant hypothesis ensures that the closed convexification is proper; an unqualified statement including arbitrary improper functions would need separate conventions.

Here are the essential proof steps. The Fenchel–Young inequality gives $f^{\ast \ast }\le f$. Every term in the supremum defining $f^{\ast \ast }$ is an affine minorant of $f$, and conversely any affine minorant $⟨p,x⟩+a\le f(x)$ has $a\le -f^{\ast }(p)$. Thus $f^{\ast \ast }$ is exactly the supremum of all affine minorants, hence is convex and lower semicontinuous.

Put $g=\mathrm{clconv}f$. Its epigraph is the closed convex hull of the epigraph of $f$. Applying the half-space representation of a closed convex set in ${\mathbb{R}}^{n+1}$ recovers that epigraph from its containing half-spaces. A containing half-space writtenhas $q\le 0$, since epigraphs extend upwards. If $q<0$, it is precisely the epigraph inequality of an affine minorant, $t\ge (⟨v,x⟩-a)/(-q)$.

Vertical half-spaces with $q=0$ must also be accounted for. Choose one affine minorant ${\ell }_0$ of $g$, which exists because $g$ is proper and closed: strictly separate $(x_0,g(x_0)-1)$ from its epigraph for a finite domain point $x_0$; the separating coefficient of $t$ cannot be zero, since that would not distinguish points with the same $x_0$. If a vertical containing inequality is $⟨v,x⟩\le a$, thenis still an affine minorant on the domain of $g$. At a point violating the vertical inequality, these minorants tend to infinity as $L\to \infty$. Thus vertical domain restrictions are also recovered by the supremum of affine minorants.

Consequently $g$ equals that supremum. Every affine minorant of $g$ is below $f$, while the epigraph of any affine minorant of $f$ contains its closed convex epigraph hull. Therefore $f$ and $g$ have the same affine minorants, completing $g=f^{\ast \ast }$. The theorem in the previous solution supplies the geometric separation step behind biconjugation.

One form of Farkas' lemma states that exactly one of the following holds:Their mutual exclusion is immediate: if both held, $b^{T}y=z^T{A}^{T}y\ge 0$.

For existence of the alternative certificate, let $C=\{Az:z\ge 0\}$, the finitely generated cone of the columns of $A$. It is convex. It is also closed, a fact that must be justified rather than assumed for arbitrary linear images of closed cones. In a representation with dependent active generators, choose a nonzero dependence ${\sum }_j{d}_j{a}_j=0$ with at least one $d_j>0$. Subtractfrom the nonnegative coefficient vector. The represented point is unchanged, all coefficients remain nonnegative, and at least one active coefficient disappears. Iteration produces a representation with independent active columns. For a convergent sequence in $C$, pass to a subsequence using the same independent set, possible because there are finitely many sets. Its coefficients converge through a fixed left inverse, and their limits remain nonnegative. This proves closedness of finitely generated cones.

The indicator functional ${\delta }_C$ is therefore proper, lower semicontinuous and convex. Its Legendre-Fenchel transform isIndeed a positive pairing with a cone generator can be scaled arbitrarily, while all nonpositive pairings give supremum zero. The Fenchel-Moreau theorem now givesIf $b\notin C$, the left side is infinite, so some feasible $w$ has $⟨w,b⟩>0$. Taking $y=-w$ gives $A^{T}y\ge 0$ and $b^{T}y<0$. If $b\in C$, the first alternative holds. The biconjugation theorem applied to a closed finitely generated cone proves the alternative.

For inequalities $Cx\le d$ with unrestricted $x$, split $x=x_{+}-x_{-}$ and add nonnegative slack:The equivalent Farkas certificate for linear inequalities is

LetThe proximal operator optimality condition saysBy subgradient inversion under convex conjugacy, which uses Fenchel–Young inequality and $f^{\ast \ast }=f$ from the Fenchel-Moreau theorem,Consequently $z-p=\tau v\in \tau \partial f^{\ast }(p)$, meaning $p=B_{\tau f^{\ast }}(z)$. Uniqueness of the backward subgradient step identifies the result:This is the scaled Moreau decomposition. It requires one backward step on $f$, with reciprocal parameter $1/\tau$ and scaled input $z/\tau$, followed by a scalar multiplication and subtraction. No separate proximal computation of the convex conjugate is needed.

For an extended-real function define its Fenchel conjugate by $f^{\ast }(p)={\sup }_x(⟨p,x⟩-f(x))$ and its biconjugate by $f^{\ast \ast }(x)={\sup }_p(⟨p,x⟩-f^{\ast }(p))$. The Fenchel-Moreau theorem states, in the standard proper-envelope setting,Here the right side is the largest lower semicontinuous convex function below $f$, equivalently the function whose epigraph is the closed convex hull of $\mathrm{epi}f$. It is enough to assume $f$ is proper and has an affine minorant, ensuring this envelope is proper. In particular, for a proper convex function that is lower semicontinuous, $\boxed{f^{\ast \ast }=f}$.

First, $⟨p,x⟩-f^{\ast }(p)\le f(x)$ by the definition of the convex conjugate. The biconjugate is a supremum of continuous affine functions, so it is convex, lower semicontinuous and no greater than $f$. Second, the best intercept for an affine minorant with slope $p$ is $-f^{\ast }(p)$: $⟨p,x⟩+a\le f(x)$ for all $x$ precisely when $a\le -f^{\ast }(p)$. Thus $f^{\ast \ast }$ is the supremum of all affine minorants.

To prove that no part of the closed convex envelope is missed, set $E=\mathrm{cl}\mathrm{conv}(\mathrm{epi}f)$. The half-space representation of a closed convex set from part (a), applied in ${\mathbb{R}}^{n+1}$, separates any $(x_0,t_0)\notin E$ from $E$ by an inequality $a\cdot x+bt\le d$. Because $E$ is upward closed, $b\le 0$. If $b<0$, division by $-b$ gives an affine minorant $\ell$ with $\ell (x_0)>t_0$.

A vertical separator has $b=0$. Let ${\ell }_0$ be an existing affine minorant. Combine $a\cdot x\le d$ with ${\ell }_0(x)\le t$ to obtainSince $a\cdot x_0-d>0$, a sufficiently small positive $\epsilon$ makes this affine function exceed $t_0$. Thus vertical half-spaces can be approximated by nonvertical epigraph supports. Every point below the envelope is excluded by an affine minorant, so the supremum of these minorants is exactly the envelope. This is the decisive use of part (a).

Properness and the minorant convention matter for unrestricted extended-real functions. For example, $f(x)=-x^2$ on $\mathbb{R}$ has no affine minorant; $f^{\ast }\equiv +\infty$ and $f^{\ast \ast }\equiv -\infty$. With the corresponding improper-envelope convention its closed convex envelope is also $-\infty$. The theorem should not silently describe such an envelope as proper. The identically $+\infty$ function is another degenerate case, handled separately by extended-real conventions.

The conjugate of an infimal convolution is the sum of the conjugates:Here $g^{\ast }(p)=\frac{1}{2}\Vert p{\Vert }_2^2$, and $h^{\ast }(p)={\delta }_{[-1,1{]}^n}(p)$, the indicator functional of the unit infinity-norm ball. ThusThe infimal convolution is finite convex and continuous, so the Fenchel-Moreau theorem applies without a closure defect. By equality in the Fenchel–Young inequality and the subdifferential sum rule,This is precisely the variational characterization of projecting $x$ onto the cube. ConsequentlyFor the primal split, $y_i=\mathrm{clip}(x_i,-1,1)$ and $z_i=\mathrm{sign}(x_i)(|x_i|-1{)}_{+}$. These directly minimize the two scalar terms. The Huber loss is continuously differentiable, including at $x_i=\pm 1$, but its second derivative changes there. The one-dimensional sketch shows a quadratic center joined tangentially to linear tails:

Applied to a discrete gradient, the Huber gradient regularizer penalizes small slopes quadratically and large slopes linearly. Compared with pure squared-gradient smoothing it preserves large edges better; compared with pure total variation denoising it encourages small smooth variations and reduces the strong preference for piecewise-constant plateaus. It can therefore be useful for denoising signals or images containing both smooth regions and sharp transitions. It still penalizes edges and can bias their amplitude, and it does not guarantee complete elimination of staircasing in total variation denoising. The unit threshold must be scaled appropriately for data units and grid spacing.

Take $F=(F^{\ast }{)}^{\ast }$ with the usual proper closed convex assumptions, so the Fenchel-Moreau theorem identifies its conjugate with the supplied $F^{\ast }$. Put $z^k=u^k-\tau D^T{v}^k$. The first update is $w^{k+1}={\mathrm{prox}}_{F^{\ast }/\tau }(z^k/\tau )$. By Moreau decomposition, the second update isAlso $\tau (D^T{v}^k+w^{k+1})=u^k-u^{k+1}$, so the argument in the final update contains $u^{k+1}-\tau (D^T{v}^k+w^{k+1})=2u^{k+1}-u^k$. Eliminating the auxiliary variable givesThis is the primal-dual hybrid gradient method with extrapolation parameter one and the primal update performed first. Its saddle point function is $F(u)+⟨Du,v⟩-G(v)$, corresponding to the primal objective $F(u)+G^{\ast }(Du)$.

For proper lower semicontinuous convex functions and a nonempty saddle-point set in these finite-dimensional spaces, the standard convergence of primal-dual hybrid gradient theorem gives the sufficient parameter conditionHere $\Vert D{\Vert }_2$ is the operator norm, or largest singular value. For $D\ne 0$, one possible choice is $\tau =\sigma =\eta /\Vert D{\Vert }_2$ with $0<\eta <1$; for $D=0$ any positive steps satisfy the condition. The existence assumption is necessary: step sizes alone cannot guarantee convergence to a saddle point that does not exist.

For a real Banach space $X$ and a proper convex function $E:X\to \mathbb{R}\cup \{+\infty \}$, the convex conjugate is the extended-valued function on the continuous dual spaceHere proper means that $E$ never takes $-\infty$ and is finite somewhere. The pairing is the duality pairing, or the inner product after identification in a Hilbert space. The conjugate is convex and lower semicontinuous because it is a supremum of continuous affine functions. Its definition gives the Fenchel–Young inequality $E(x)+E^{\ast }(p)\ge ⟨x,p⟩$; the Fenchel-Moreau theorem recovers $E$ as $E^{\ast \ast }$ under the stated closed-convex hypotheses.

By the definition of the subdifferential, $p\in \partial J(u)$ meansRearrangement bounds $⟨v,p⟩-J(v)$ by $⟨u,p⟩-J(u)$, and equality is attained at $v=u$. Taking the supremum in the definition of the convex conjugate gives $J^{\ast }(p)=⟨u,p⟩-J(u)$. Conversely this equality bounds every member of that supremum and rearranges to the subgradient inequality. ThereforeApply the same argument to $J^{\ast }$ and use the Fenchel-Moreau theorem $J^{\ast \ast }=J$. With the canonical Hilbert identification of the bidual, the equality is also equivalent to $u\in \partial J^{\ast }(p)$. This proves subgradient inversion under convex conjugacy:The conditions include finiteness at the points in question; expressions involving $+\infty$ are not subgradients merely by formal subtraction.

For a proper lower-semicontinuous convex functional on a real Hilbert space, $p\in \partial J(u)$ means $⟨v,p⟩-J(v)\le ⟨u,p⟩-J(u)$ for all $v$. Taking the supremum gives equality in the Fenchel–Young inequality: $J(u)+J^{\ast }(p)=⟨u,p⟩$. Apply the same argument to $J^{\ast }$ and use the Fenchel-Moreau theorem $J^{\ast \ast }=J$ to obtain the reciprocal subgradient condition.