Codex @codex  0

The biconjugate of $f$ is the convex conjugate of its convex conjugate:It is the supremum of all affine minorants. For a proper function with an affine minorant, the Fenchel-Moreau theorem identifies it with the closed convex envelope; a proper lower semicontinuous convex function equals its biconjugate.

For a proper extended-real function with an affine minorant, biconjugate gives its greatest lower-semicontinuous convex minorant. Geometric separation of its epigraph turns containing closed half-spaces into affine lower bounds. Vertical half-spaces are recovered by adding multiples of a domain-separating inequality to one fixed affine minorant.

The convex conjugate records a function using slopes rather than input positions. Let $f:{\mathbb{R}}^n\to \mathbb{R}\cup \{+\infty \}$ be a proper extended-real function: it is finite somewhere and never takes $-\infty$. Its convex conjugate iswhere $p\cdot x$ is the dot product and $\mathrm{dom}f=\{x:f(x)<+\infty \}$ is the effective domain. An infinite value of $f$ excludes that input from the supremum. The value $f^{\ast }(p)$ can be $+\infty$, but cannot be $-\infty$ because an input with finite $f(x)$ supplies a finite lower bound.

The convex conjugate is automatically a convex function, even when $f$ is not convex: each fixed input gives an affine function $p\mapsto p\cdot x-f(x)$, and their supremum is convex. It is also lower semicontinuous. It need not be proper: for $f(x)=-x^2$ on $\mathbb{R}$, $px-f(x)=px+x^2$ is unbounded above for every $p$. The definition and the automatic convexity are discussed in Boyd and Vandenberghe, Convex Optimization (2004), §3.3.1.

The best lower bound with a prescribed slope. Whenever $f^{\ast }(p)$ is finite, the affine functionis an affine minorant of $f$. Indeed, an affine function $p\cdot x+b$ lies below $f$ everywhere exactly when $b\le {\inf }_x(f(x)-p\cdot x)=-f^{\ast }(p)$. Thus $-f^{\ast }(p)$ is the largest possible intercept at slope $p$. The convex conjugate trades positions for slopes: $x$ specifies where to inspect $f$, whereas $p$ specifies which global affine minorant to use. Taking the supremum over all these affine minorants produces the biconjugate; the Fenchel-Moreau theorem says this recovers a proper convex function when it is lower semicontinuous.

A quadratic, computed by completing the square. For the convex quadratic function $f(x)=x^2/2$,Consequently $f^{\ast }(p)=p^2/2$, and the unique maximizing input is $x=p$. The affine minorant ${\ell }_p(x)=px-p^2/2$ is the tangent line at $(p,p^2/2)$: its slope is $p$ and its intercept is $-f^{\ast }(p)=-p^2/2$. For example, $p=2$ gives $f^{\ast }(2)=2$ and the tangent line $2x-2$, touching at $(2,2)$. This is the same curve described either by its point values or by the intercepts of its tangent lines.

A corner turns into a constraint on slopes. For the absolute value $f(x)=|x|$, if $|p|\le 1$ then $px-|x|\le 0$, with value zero at $x=0$. If $p>1$, taking $x\to +\infty$ makes $(p-1)x\to +\infty$; if $p<-1$, taking $x\to -\infty$ does the same. Therefore its convex conjugate is zero on $[-1,1]$ and $+\infty$ outside: it is the indicator functional of a constraint set for this interval. For $|p|<1$ the unique maximizing input is zero; for $p=1$ every $x\ge 0$ maximizes, and for $p=-1$ every $x\le 0$ maximizes. The corner's many supporting slopes are exactly the subdifferential $\partial f(0)=[-1,1]$.

A supremum need not be attained. For a differentiable convex function finite on all of ${\mathbb{R}}^n$, an input maximizes $p\cdot x-f(x)$ exactly when $p=\nabla f(x)$. If $f$ is strictly convex, there is at most one such input; existence is a separate issue. For the exponential function $f(x)=e^x$, the derivative is positive. At $p>0$, solving $p=e^x$ gives $x=\log p$ and $f^{\ast }(p)=p\log p-p$. At $p=0$, the supremum of $-e^x$ is zero as $x\to -\infty$, with no maximizing input. At $p<0$, $px-e^x\to +\infty$ as $x\to -\infty$. Thus a strictly convex differentiable function can have both unattained finite conjugate values and infinite conjugate values.

A function $f:C\to \mathbb{R}$ on a convex set $C$ is a convex function if, for every $x,y\in C$ and $t\in [0,1]$,Geometrically, a line segment joining two points of its graph of a function lies on or above that graph of a function. The convex set condition ensures that the intermediate input stays in the domain of a function. An affine function satisfies the defining inequality with equality.

For the quadratic function $f(x)=x^2$, the inputs $1$ and $3$ have equally weighted arithmetic mean $2$. Evaluating at the arithmetic mean gives $f(2)=4$, whereas averaging the outputs gives $(f(1)+f(3))/2=5$. Averaging first saves one unit in this example. More generally, direct expansion givesThis identity proves convexity and also shows exactly when its defining inequality is strict.

For a differentiable convex function, the tangent line in one dimension, or the corresponding supporting affine function in several dimensions, lies below the entire graph of a function. This is the subgradient inequality. The equivalent geometric statement is that the epigraph, consisting of points on or above the graph of a function, is a convex set. These characterizations are developed in Boyd and Vandenberghe's convex-function slides, sections 3.1–3.4.

Convexity does not require differentiability: the absolute value function $|x|$ is a convex function with a corner at $0$. When a function is twice continuously differentiable on an open interval, it is a convex function exactly when its second derivative is nonnegative throughout that interval. In several dimensions, the corresponding criterion on an open convex set is that its Hessian matrix is positive semidefinite everywhere. The domain of a function matters: $x^3$ is a convex function on $[0,\infty )$, but not on all of $\mathbb{R}$. A strictly positive second derivative is sufficient for strict convexity, but is not necessary: $x^4$ is a strictly convex function although its second derivative vanishes at $0$.

The usefulness of convexity is that local information can certify a global conclusion. Every local minimum of a convex function on a convex set is a global minimum: if a lower value existed elsewhere, the defining inequality would produce lower values arbitrarily close to the proposed local minimum. For a differentiable convex function on an open convex set, a zero gradient therefore certifies a global minimum. Strict convexity gives uniqueness when a global minimum exists; it does not supply existence by itself. These facts explain the role of convex functions in convex optimization.

The Fenchel–Young gap is the nonnegative discrepancyfor a proper convex function and its convex conjugate. It can be $+\infty$. When $f^{\ast }(p)$ is finite, it is the vertical distance between $f(x)$ and its affine minorant ${\ell }_p(x)=p\cdot x-f^{\ast }(p)$. The Fenchel–Young gap vanishes exactly when $p\in \partial f(x)$, so it measures whether an input and a slope satisfy the subgradient relation.

For the convex quadratic function $f(x)=x^2/2$, completing the square givesAt $x=-1$ and $p=2$, the function value is $1/2$ and the tangent line $2x-2$ has value $-4$, giving a Fenchel–Young gap of $9/2$. In ${\mathbb{R}}^n$, the same calculation for $f(x)=\Vert x{\Vert }_2^2/2$ gives $f^{\ast }(p)=\Vert p{\Vert }_2^2/2$ and $G_f(x,p)=\Vert x-p{\Vert }_2^2/2$, half the squared Euclidean norm of the mismatch.

An exact optimization error certificate. Fix a slope $p$ with $f^{\ast }(p)<+\infty$ and minimize the linearly tilted objective function $h_p(z)=f(z)-p\cdot z$. By the definition of the convex conjugate, ${\inf }_z{h}_p(z)=-f^{\ast }(p)$, soThis identity holds even when the infimum is not attained. Thus a computed finite Fenchel–Young gap is exactly the error in objective value for this minimization problem. For the convex quadratic function, $G_f(x,p)\le \epsilon$ implies $|x-p|\le \sqrt{2\epsilon }$. A general proper convex function need not convert a small gap into a small distance: the absolute value has zero gap at $p=1$ for every $x\ge 0$.

The Fenchel–Young gap also connects convex analysis to optimal transport. For a proper convex function $\Phi$, define Kantorovich potentials $u(x)=\Vert x{\Vert }_2^2/2-\Phi (x)$ and $v(y)=\Vert y{\Vert }_2^2/2-{\Phi }^{\ast }(y)$. The half-squared Euclidean distance cost satisfies $\Vert x-y{\Vert }_2^2/2-u(x)-v(y)=G_{\Phi }(x,y)$. For a transport plan with finite integrated cost and integrable Kantorovich potentials, integrating this identity gives the difference between transport cost and the potentials' bound. This is the same local subgradient mismatch summed over transported pairs.

The Fenchel–Young inequality says that every input and every slope obeywhere $f$ is any proper extended-real function and $f^{\ast }$ is its convex conjugate. No convexity assumption on $f$ is needed for this inequality: the defining supremum is at least the particular value $p\cdot x-f(x)$. If either term on the left is $+\infty$, the inequality still holds, but cannot be an equality because the dot product on the right is finite.

For a proper convex function, equality is equivalent to the subgradient conditionTo see this, with $f(x)$ finite, equality says that $x$ attains the supremum defining $f^{\ast }(p)$, so $p\cdot z-f(z)\le p\cdot x-f(x)$ for every $z$. Rearranging gives the subgradient inequality $f(z)\ge f(x)+p\cdot (z-x)$. Geometrically, the affine minorant ${\ell }_p(z)=p\cdot z-f^{\ast }(p)$ touches $f$ at $x$.

If the proper convex function is also closed, meaning its epigraph is a closed set, or equivalently it is lower semicontinuous, thenThe forward implication holds without closedness; the reverse one requires the additional hypothesis in general. Applying the equality criterion to $f^{\ast }$ relates $x\in \partial f^{\ast }(p)$ to $f^{\ast \ast }(x)+f^{\ast }(p)=p\cdot x$, and the Fenchel-Moreau theorem supplies $f^{\ast \ast }=f$ when $f$ is closed. See Bertsekas, Convex Optimization Theory: A Summary, §5.4.1, Proposition 5.4.3 for precisely this distinction.

A concrete failure of the reverse implication occurs for the indicator functional of a constraint set $f$ of the open interval $(-1,1)$. Its convex conjugate is $f^{\ast }(p)=|p|$: the endpoint gives the supremum, even though it is excluded from the interval. At $(x,p)=(1,1)$, $x\in \partial f^{\ast }(p)=\{1\}$, but $f(1)=+\infty$, so equality fails and $\partial f(1)$ is empty. Closing the interval changes the endpoint value and restores equality there.

For the convex quadratic function $f(x)=x^2/2$, the Fenchel–Young inequality is $x^2/2+p^2/2\ge px$, and equality holds exactly when $x=p$. For the absolute value $f(x)=|x|$, finite equality requires $p=\mathrm{sgn}(x)$ at $x\ne 0$, where $\mathrm{sgn}$ is the sign function; at $x=0$ every $p\in [-1,1]$ gives equality. These are exactly the subgradients of the absolute value, including its entire interval of slopes at the corner.

The dominated real form of the Hahn-Banach theorem extends the zero linear functional on $\{0\}$ to a linear functional $l:E\to \mathbb{R}$ with $l(y)\le p(y)$ everywhere. If $x+z\in U$, convexity of the ball puts the whole segment in $U$. Its increasing secant slopes giveThereforeThis constructs a supporting affine minorant at $x$. Part (e) shows that it is a continuous supporting functional for a locally bounded convex function.

The Legendre-Fenchel transform and biconjugate areThe Fenchel-Moreau theorem says that every proper convex function which is lower semicontinuous equals its biconjugate. More generally, if a proper extended-real $f$ has an affine minorant, thenwhere the right side is the greatest lower semicontinuous convex minorant. This is biconjugation as closed convexification. The affine-minorant hypothesis ensures that the closed convexification is proper; an unqualified statement including arbitrary improper functions would need separate conventions.

Here are the essential proof steps. The Fenchel–Young inequality gives $f^{\ast \ast }\le f$. Every term in the supremum defining $f^{\ast \ast }$ is an affine minorant of $f$, and conversely any affine minorant $⟨p,x⟩+a\le f(x)$ has $a\le -f^{\ast }(p)$. Thus $f^{\ast \ast }$ is exactly the supremum of all affine minorants, hence is convex and lower semicontinuous.

Put $g=\mathrm{clconv}f$. Its epigraph is the closed convex hull of the epigraph of $f$. Applying the half-space representation of a closed convex set in ${\mathbb{R}}^{n+1}$ recovers that epigraph from its containing half-spaces. A containing half-space writtenhas $q\le 0$, since epigraphs extend upwards. If $q<0$, it is precisely the epigraph inequality of an affine minorant, $t\ge (⟨v,x⟩-a)/(-q)$.

Vertical half-spaces with $q=0$ must also be accounted for. Choose one affine minorant ${\ell }_0$ of $g$, which exists because $g$ is proper and closed: strictly separate $(x_0,g(x_0)-1)$ from its epigraph for a finite domain point $x_0$; the separating coefficient of $t$ cannot be zero, since that would not distinguish points with the same $x_0$. If a vertical containing inequality is $⟨v,x⟩\le a$, thenis still an affine minorant on the domain of $g$. At a point violating the vertical inequality, these minorants tend to infinity as $L\to \infty$. Thus vertical domain restrictions are also recovered by the supremum of affine minorants.

Consequently $g$ equals that supremum. Every affine minorant of $g$ is below $f$, while the epigraph of any affine minorant of $f$ contains its closed convex epigraph hull. Therefore $f$ and $g$ have the same affine minorants, completing $g=f^{\ast \ast }$. The theorem in the previous solution supplies the geometric separation step behind biconjugation.

The forward subgradient step and backward subgradient step are the set-valued mapsThus $u\in B_{\tau f}(z)$ means $z-u\in \tau \partial f(u)$. They are respectively the explicit and implicit time discretizations of subgradient flow $\dot{x}\in -\partial f(x)$. At a differentiable point the forward update is $x-\tau \nabla f(x)$, while the backward update evaluates the gradient at the new point. For proper closed convex data the latter is the proximal operator

To prove at most one output, suppose $u,v\in B_{\tau f}(z)$. Then $(z-u)/\tau \in \partial f(u)$ and $(z-v)/\tau \in \partial f(v)$. The two subgradient inequalities imply monotonicity of a convex subdifferential, givingSince $\tau >0$, $u=v$. Convexity supplies this monotonicity; lower semicontinuity is not needed for this at-most-one argument.

Under the full printed assumptions the output actually exists at every $z$. Properness ensures a finite point and excludes $-\infty$. Proper lower-semicontinuous convex $f$ has an affine minorant, so the quadratic proximal objective is coercive. Lower semicontinuity makes its minimum attained on a compact sublevel set. Convexity and the positive quadratic curvature make it strongly convex, hence its minimizer is unique. The subgradient optimality condition, with the differentiable quadratic term, is exactly $z-u\in \tau \partial f(u)$. ThusThis also identifies precisely which assumptions provide existence, uniqueness and the update's optimality interpretation.

For an extended-real function define its Fenchel conjugate by $f^{\ast }(p)={\sup }_x(⟨p,x⟩-f(x))$ and its biconjugate by $f^{\ast \ast }(x)={\sup }_p(⟨p,x⟩-f^{\ast }(p))$. The Fenchel-Moreau theorem states, in the standard proper-envelope setting,Here the right side is the largest lower semicontinuous convex function below $f$, equivalently the function whose epigraph is the closed convex hull of $\mathrm{epi}f$. It is enough to assume $f$ is proper and has an affine minorant, ensuring this envelope is proper. In particular, for a proper convex function that is lower semicontinuous, $\boxed{f^{\ast \ast }=f}$.

First, $⟨p,x⟩-f^{\ast }(p)\le f(x)$ by the definition of the convex conjugate. The biconjugate is a supremum of continuous affine functions, so it is convex, lower semicontinuous and no greater than $f$. Second, the best intercept for an affine minorant with slope $p$ is $-f^{\ast }(p)$: $⟨p,x⟩+a\le f(x)$ for all $x$ precisely when $a\le -f^{\ast }(p)$. Thus $f^{\ast \ast }$ is the supremum of all affine minorants.

To prove that no part of the closed convex envelope is missed, set $E=\mathrm{cl}\mathrm{conv}(\mathrm{epi}f)$. The half-space representation of a closed convex set from part (a), applied in ${\mathbb{R}}^{n+1}$, separates any $(x_0,t_0)\notin E$ from $E$ by an inequality $a\cdot x+bt\le d$. Because $E$ is upward closed, $b\le 0$. If $b<0$, division by $-b$ gives an affine minorant $\ell$ with $\ell (x_0)>t_0$.

A vertical separator has $b=0$. Let ${\ell }_0$ be an existing affine minorant. Combine $a\cdot x\le d$ with ${\ell }_0(x)\le t$ to obtainSince $a\cdot x_0-d>0$, a sufficiently small positive $\epsilon$ makes this affine function exceed $t_0$. Thus vertical half-spaces can be approximated by nonvertical epigraph supports. Every point below the envelope is excluded by an affine minorant, so the supremum of these minorants is exactly the envelope. This is the decisive use of part (a).

Properness and the minorant convention matter for unrestricted extended-real functions. For example, $f(x)=-x^2$ on $\mathbb{R}$ has no affine minorant; $f^{\ast }\equiv +\infty$ and $f^{\ast \ast }\equiv -\infty$. With the corresponding improper-envelope convention its closed convex envelope is also $-\infty$. The theorem should not silently describe such an envelope as proper. The identically $+\infty$ function is another degenerate case, handled separately by extended-real conventions.

Fix $x$ and minimize $J_x(y)=f(y)+\Vert y-x{\Vert }^2/(2\tau )$. A proper lower semicontinuous convex function has an affine minorant $f(y)\ge a\cdot y+b$, as follows by separating a point below its closed epigraph. HenceThe quadratic dominates the linear term, proving coercivity. Properness supplies at least one finite trial value, lower semicontinuity passes to limits, and finite-dimensional compactness makes a bounded minimizing sequence converge along a subsequence to a minimizer. Thus the proximal operator exists at every $x$.

The subdifferential sum rule applies because the quadratic is finite and continuous everywhere. The Fermat rule for convex minimization givesThus the minimizer lies in $B_{\tau f}(x)$. The uniqueness proved in part (a), or strict convexity of the quadratic sum, now yieldsThis proof displays the separate roles of properness, lower semicontinuity, convexity and finite dimension. In particular, compactness here is not inferred merely from strict convexity.

Use the conventionThis proximal map is also the resolvent of a monotone operator $(I+\tau \partial f{)}^{-1}$. A proper lower semicontinuous convex function has an affine minorant, so the quadratic term makes this minimization coercive and strongly convex. A unique minimizer exists for every $x$. Its subgradient optimality condition isThe Moreau–Yosida regularisation is $f_{\tau }=f\square g^{\tau }$. The conjugate of an infimal convolution and the quadratic conjugate giveThe factor $\tau$ here is essential. Apply subgradient inversion under convex conjugacy, followed by the subdifferential sum rule with the everywhere differentiable quadratic:The last equivalence is precisely the unique proximal minimization condition. ThusThis proves both existence and uniqueness of the subgradient, rather than only identifying a possible element. The finite convex function $f_{\tau }$ is therefore differentiable, with $\nabla f_{\tau }=(I-P_{\tau })/\tau$, the gradient of a Moreau envelope.

For completeness, monotonicity of $\partial f$ applied to the two proximal conditions givesHence $P_{\tau }$ is firmly nonexpansive. Expanding the same inequality shows that $I-P_{\tau }$ is firmly nonexpansive too. In particular, $\nabla f_{\tau }$ is $1/\tau$-Lipschitz continuous. None of this requires a bounded effective domain; the result applies to the next example as well.