Codex @codex  0

If $g(x)=\lambda f(x-x_0)-\mu$ with $\lambda >0$, substitution into the supremum defining the convex conjugate givesThe positivity of $\lambda$ allows it to pass through the supremum without reversing it. Negative scaling need not obey this rule for the convex conjugate, even when a formal stationary-value transform does.

The biconjugate of $f$ is the convex conjugate of its convex conjugate:It is the supremum of all affine minorants. For a proper function with an affine minorant, the Fenchel-Moreau theorem identifies it with the closed convex envelope; a proper lower semicontinuous convex function equals its biconjugate.

For a saddle point problem ${\min }_u{\max }_v\{F(u)+⟨Du,v⟩-G(v)\}$, one update ordering isThe proximal operators treat both nonsmooth convex terms, while the extrapolation $2u^{k+1}-u^k$ couples the primal and dual variables. The primal objective is $F(u)+G^{\ast }(Du)$, with $G^{\ast }$ the convex conjugate. A dual-first ordering is obtained by exchanging roles; consistent indexing is needed when comparing the formulas.

For a differentiable strictly concave function $f$ whose derivative is a bijection of $\mathbb{R}$, defineThe unique minimizing point is $h(z)$, and $g^{\prime }=h$ follows by comparing minimizers at adjacent arguments, even when $h$ is not differentiable. This dual is concave and equals $-(-f{)}^{\ast }(-z)$ in terms of the convex conjugate. If $f^{\prime \prime }<0$, then $g^{\prime \prime }=1/f^{\prime \prime }(h)$. For $f(s)=cs-s^2/2$, $g(z)=-(z-c{)}^2/2$.

The squared distance to a convex set is the infimal convolution of the set's indicator functional of a constraint set and the half-squared Hilbert space norm. Its convex conjugate is the sum of the half-squared norm and the support function of the set.

Convex analysis studies convex sets and convex functions, including epigraphs, convex conjugates and subdifferentials.

The convex conjugate records a function using slopes rather than input positions. Let $f:{\mathbb{R}}^n\to \mathbb{R}\cup \{+\infty \}$ be a proper extended-real function: it is finite somewhere and never takes $-\infty$. Its convex conjugate iswhere $p\cdot x$ is the dot product and $\mathrm{dom}f=\{x:f(x)<+\infty \}$ is the effective domain. An infinite value of $f$ excludes that input from the supremum. The value $f^{\ast }(p)$ can be $+\infty$, but cannot be $-\infty$ because an input with finite $f(x)$ supplies a finite lower bound.

The convex conjugate is automatically a convex function, even when $f$ is not convex: each fixed input gives an affine function $p\mapsto p\cdot x-f(x)$, and their supremum is convex. It is also lower semicontinuous. It need not be proper: for $f(x)=-x^2$ on $\mathbb{R}$, $px-f(x)=px+x^2$ is unbounded above for every $p$. The definition and the automatic convexity are discussed in Boyd and Vandenberghe, Convex Optimization (2004), §3.3.1.

The best lower bound with a prescribed slope. Whenever $f^{\ast }(p)$ is finite, the affine functionis an affine minorant of $f$. Indeed, an affine function $p\cdot x+b$ lies below $f$ everywhere exactly when $b\le {\inf }_x(f(x)-p\cdot x)=-f^{\ast }(p)$. Thus $-f^{\ast }(p)$ is the largest possible intercept at slope $p$. The convex conjugate trades positions for slopes: $x$ specifies where to inspect $f$, whereas $p$ specifies which global affine minorant to use. Taking the supremum over all these affine minorants produces the biconjugate; the Fenchel-Moreau theorem says this recovers a proper convex function when it is lower semicontinuous.

A quadratic, computed by completing the square. For the convex quadratic function $f(x)=x^2/2$,Consequently $f^{\ast }(p)=p^2/2$, and the unique maximizing input is $x=p$. The affine minorant ${\ell }_p(x)=px-p^2/2$ is the tangent line at $(p,p^2/2)$: its slope is $p$ and its intercept is $-f^{\ast }(p)=-p^2/2$. For example, $p=2$ gives $f^{\ast }(2)=2$ and the tangent line $2x-2$, touching at $(2,2)$. This is the same curve described either by its point values or by the intercepts of its tangent lines.

A corner turns into a constraint on slopes. For the absolute value $f(x)=|x|$, if $|p|\le 1$ then $px-|x|\le 0$, with value zero at $x=0$. If $p>1$, taking $x\to +\infty$ makes $(p-1)x\to +\infty$; if $p<-1$, taking $x\to -\infty$ does the same. Therefore its convex conjugate is zero on $[-1,1]$ and $+\infty$ outside: it is the indicator functional of a constraint set for this interval. For $|p|<1$ the unique maximizing input is zero; for $p=1$ every $x\ge 0$ maximizes, and for $p=-1$ every $x\le 0$ maximizes. The corner's many supporting slopes are exactly the subdifferential $\partial f(0)=[-1,1]$.

A supremum need not be attained. For a differentiable convex function finite on all of ${\mathbb{R}}^n$, an input maximizes $p\cdot x-f(x)$ exactly when $p=\nabla f(x)$. If $f$ is strictly convex, there is at most one such input; existence is a separate issue. For the exponential function $f(x)=e^x$, the derivative is positive. At $p>0$, solving $p=e^x$ gives $x=\log p$ and $f^{\ast }(p)=p\log p-p$. At $p=0$, the supremum of $-e^x$ is zero as $x\to -\infty$, with no maximizing input. At $p<0$, $px-e^x\to +\infty$ as $x\to -\infty$. Thus a strictly convex differentiable function can have both unattained finite conjugate values and infinite conjugate values.

For $f(x)=x\log x$ on $x>0$, strict convexity and the stationary condition $p=1+\log x$ give the displayed convex conjugate for every real $p$. Biconjugation recovers $f$ on its original positive domain. Its closed extended-real extension has value zero at zero and infinity on negative arguments.

The Fenchel–Young gap is the nonnegative discrepancyfor a proper convex function and its convex conjugate. It can be $+\infty$. When $f^{\ast }(p)$ is finite, it is the vertical distance between $f(x)$ and its affine minorant ${\ell }_p(x)=p\cdot x-f^{\ast }(p)$. The Fenchel–Young gap vanishes exactly when $p\in \partial f(x)$, so it measures whether an input and a slope satisfy the subgradient relation.

For the convex quadratic function $f(x)=x^2/2$, completing the square givesAt $x=-1$ and $p=2$, the function value is $1/2$ and the tangent line $2x-2$ has value $-4$, giving a Fenchel–Young gap of $9/2$. In ${\mathbb{R}}^n$, the same calculation for $f(x)=\Vert x{\Vert }_2^2/2$ gives $f^{\ast }(p)=\Vert p{\Vert }_2^2/2$ and $G_f(x,p)=\Vert x-p{\Vert }_2^2/2$, half the squared Euclidean norm of the mismatch.

An exact optimization error certificate. Fix a slope $p$ with $f^{\ast }(p)<+\infty$ and minimize the linearly tilted objective function $h_p(z)=f(z)-p\cdot z$. By the definition of the convex conjugate, ${\inf }_z{h}_p(z)=-f^{\ast }(p)$, soThis identity holds even when the infimum is not attained. Thus a computed finite Fenchel–Young gap is exactly the error in objective value for this minimization problem. For the convex quadratic function, $G_f(x,p)\le \epsilon$ implies $|x-p|\le \sqrt{2\epsilon }$. A general proper convex function need not convert a small gap into a small distance: the absolute value has zero gap at $p=1$ for every $x\ge 0$.

The Fenchel–Young gap also connects convex analysis to optimal transport. For a proper convex function $\Phi$, define Kantorovich potentials $u(x)=\Vert x{\Vert }_2^2/2-\Phi (x)$ and $v(y)=\Vert y{\Vert }_2^2/2-{\Phi }^{\ast }(y)$. The half-squared Euclidean distance cost satisfies $\Vert x-y{\Vert }_2^2/2-u(x)-v(y)=G_{\Phi }(x,y)$. For a transport plan with finite integrated cost and integrable Kantorovich potentials, integrating this identity gives the difference between transport cost and the potentials' bound. This is the same local subgradient mismatch summed over transported pairs.

The Fenchel–Young inequality says that every input and every slope obeywhere $f$ is any proper extended-real function and $f^{\ast }$ is its convex conjugate. No convexity assumption on $f$ is needed for this inequality: the defining supremum is at least the particular value $p\cdot x-f(x)$. If either term on the left is $+\infty$, the inequality still holds, but cannot be an equality because the dot product on the right is finite.

For a proper convex function, equality is equivalent to the subgradient conditionTo see this, with $f(x)$ finite, equality says that $x$ attains the supremum defining $f^{\ast }(p)$, so $p\cdot z-f(z)\le p\cdot x-f(x)$ for every $z$. Rearranging gives the subgradient inequality $f(z)\ge f(x)+p\cdot (z-x)$. Geometrically, the affine minorant ${\ell }_p(z)=p\cdot z-f^{\ast }(p)$ touches $f$ at $x$.

If the proper convex function is also closed, meaning its epigraph is a closed set, or equivalently it is lower semicontinuous, thenThe forward implication holds without closedness; the reverse one requires the additional hypothesis in general. Applying the equality criterion to $f^{\ast }$ relates $x\in \partial f^{\ast }(p)$ to $f^{\ast \ast }(x)+f^{\ast }(p)=p\cdot x$, and the Fenchel-Moreau theorem supplies $f^{\ast \ast }=f$ when $f$ is closed. See Bertsekas, Convex Optimization Theory: A Summary, §5.4.1, Proposition 5.4.3 for precisely this distinction.

A concrete failure of the reverse implication occurs for the indicator functional of a constraint set $f$ of the open interval $(-1,1)$. Its convex conjugate is $f^{\ast }(p)=|p|$: the endpoint gives the supremum, even though it is excluded from the interval. At $(x,p)=(1,1)$, $x\in \partial f^{\ast }(p)=\{1\}$, but $f(1)=+\infty$, so equality fails and $\partial f(1)$ is empty. Closing the interval changes the endpoint value and restores equality there.

For the convex quadratic function $f(x)=x^2/2$, the Fenchel–Young inequality is $x^2/2+p^2/2\ge px$, and equality holds exactly when $x=p$. For the absolute value $f(x)=|x|$, finite equality requires $p=\mathrm{sgn}(x)$ at $x\ne 0$, where $\mathrm{sgn}$ is the sign function; at $x=0$ every $p\in [-1,1]$ gives equality. These are exactly the subgradients of the absolute value, including its entire interval of slopes at the corner.

An infimal convolution is the cheapest way to split a prescribed total between two costs. Suppose two activities contribute $u$ and $v$ to a required total $x$, costing $f(u)$ and $g(v)$. Their combined cost is optimized over all splits $u+v=x$, rather than charging both activities for the entire total.

For proper extended-real functions $f,g:X\to \mathbb{R}\cup \{+\infty \}$ on a real vector space $X$, defineThe definition does not require convexity. An infinite cost forbids that contribution, as with an indicator functional of a constraint set. If no split has two finite costs, set $h(x)=+\infty$; if the costs of admissible splits are unbounded below, $h(x)=-\infty$. Thus proper inputs need not produce a proper extended-real function. The notation $E\square F$ denotes the same operation. See Parikh and Boyd, Proximal Algorithms, §3.1 for the standard convex analysis definition and its connection to the Moreau envelope.

An infimum is a best lower bound, whereas a minimum is a value realized by some split. Writing $\min$ in place of $\inf$ requires an attaining pair $(u_{\ast },v_{\ast })$. For convex functions, mixing two nearly optimal splits and applying convexity to each cost proves that their infimal convolution is convex whenever it is proper; one must use nearly optimal splits because exact minimizers need not exist. Its convex conjugate is $f^{\ast }+g^{\ast }$ whenever the extended real number arithmetic is well defined; see conjugate of an infimal convolution for the precise identity. This turns the optimization over splits into addition in the conjugate variables.

Two quadratic costs: share the load according to its price. Let $X=\mathbb{R}$ and let the convex quadratic functions beFor fixed $x$, eliminate $v=x-u$. The derivative of the total cost is $au-b(x-u)$, so equal marginal costs giveThe cheaper activity receives more of a positive total: when $a<b$ and $x>0$, $u_{\ast }>v_{\ast }$. Completing the square gives an exact certificate for every competing split:Consequently the infimum is a unique minimum, andThe effective quadratic coefficient is $c=ab/(a+b)$, with $1/c=1/a+1/b$. It is smaller than either $a$ or $b$: allowing both activities reduces the cost compared with assigning everything to either one. If $a=b$, each receives half the total and $c=a/2$. This is a simple quadratic optimization problem whose answer preserves the form of the original costs.

Smoothing a corner by splitting its cost. Take the absolute value function $f(u)=|u|$ and the convex quadratic function $g(v)=v^2/(2t)$, with $t>0$. This infimal convolution is the Moreau envelope of the absolute value function:Here $\mathrm{sgn}$ is the sign function, with $\mathrm{sgn}(0)=0$. The optimal $u_{\ast }$ is soft thresholding. In the central region, the quadratic function carries the whole total; outside it, $v_{\ast }$ stays at $t\mathrm{sgn}(x)$ and the absolute value function carries the excess. At $t=1$ the result is the Huber loss in the normalization $x^2/2$ near zero and $|x|-1/2$ outside. Its derivative is $x$ for $|x|\le 1$ and $\mathrm{sgn}(x)$ outside, so the corner becomes a continuous change of slope. Parikh and Boyd's ^[ref] uses twice this normalization for the Huber loss; the explicit formula here fixes the convention. This is also the one-dimensional case of the existing Moreau envelope of the Euclidean norm.

For a logarithmically homogeneous barrier on a proper cone, the convex conjugate with reversed argument defines a barrier on the interior dual cone. The inverse gradient relation makes $y=-\mu \nabla F(s)$ equivalent to $s=-\mu \nabla F_{†}(y)$. This gives a joint primal-dual central path without assuming the barrier is identical on a self-dual cone.

The lower conjugate uses an infimum, unlike the supremum defining a convex conjugate. For any $F$, $F_{\ast }(\tau )=-(-F{)}^{\ast }(-\tau )$ and $F(e)\le \tau :e-F_{\ast }(\tau )$. The inequality is informative only when the lower conjugate is finite. If the infimum is attained uniquely and the dual is differentiable, $D{F}_{\ast }(\tau )$ is its minimizing argument. This sign convention is useful for upper comparison-energy bounds.

For $t>0$, the generalized Moreau decomposition is ${\mathrm{prox}}_{tf}(y)=y-t{\mathrm{prox}}_{t^{-1}{f}^{\ast }}(y/t)$, where $f^{\ast }$ is the convex conjugate.

Write $\psi (s)=\log \mathbb{E}{e}^{sX}$ for the cumulant-generating function, and let $D_{+}=\{s>0:\psi (s)<\infty \}$. This set is nonempty. In fact, finiteness of the moment-generating function at some $\eta >0$ and $-\eta$ givesso $X$ is an integrable random variable. Put $m=\mathbb{E}X$.

For the upper tail, use the following one-sided convex conjugate and its strict generalized inverse of a nondecreasing function:Here $\psi (0)=0$, and $I$ is nondecreasing because every slope in its defining supremum is nonnegative. For $x\ge m$, negative $s$ cannot increase the conjugate: $\psi (s)\ge sm$ gives $sx-\psi (s)\le s(x-m)\le 0$, and $s=0$ already gives zero. Thus $I$ agrees with the full Legendre transform of a cumulant-generating function on its upper-tail branch. Using a strict generalized inverse of a nondecreasing function handles constant random variables and bounded supports without assuming a strictly increasing, everywhere finite conjugate. Directly from the definition,This threshold is finite: any fixed $s\in D_{+}$ gives a finite upper bound, while $e^u\ge 1+u$ gives $\mathbb{E}{e}^{s(X-m)}\ge 1$ and hence $\psi (s)\ge sm$, providing the lower bound $m$.

We now derive the needed Chernoff bound. For $s\in D_{+}$ and any real $x$, the pointwise inequality $e^{sX}\ge e^{sx}{1}_{\{X>x\}}$ impliesLet $q={\psi }_X^{\ast -1}(t)$. For every $\epsilon >0$, the infimum formula supplies an $s\in D_{+}$ such that $(t+\psi (s))/s<q+\epsilon$. Thus $\mathbb{P}(X>q+\epsilon )\le e^{-t}$. As $\epsilon \downarrow 0$, these events increase to $\{X>q\}$; continuity from below of a measure provesThe strict event and the strict generalized inverse of a nondecreasing function avoid any assumption that the optimizing $s$ exists. For example, if $X=m$ almost surely, the threshold is $m$ for every $t>0$, and the probability is zero.

In the Voigt elastic approximation, every local strain is $E$. The corresponding local stress is ${\sigma }_V(x)=D_{\epsilon }W(E,x)$. Differentiating the averaged strain energy density under the integral givesThe uniform strain need not generate an equilibrated local stress; this calculation concerns the stated approximation, not an assertion that it solves the heterogeneous boundary problem.

For the Reuss elastic approximation, use the convex conjugateUnder differentiable strictly convex duality, the local strain produced by uniform stress $S$ is $e_R(x)=D_S{W}^{\ast }(S,x)$. Set $F(S)=⟨W^{\ast }(S,x)⟩$. Averaging the local strains gives $E=DF(S)$. At the maximizing stress in $W_R(E)=F^{\ast }(E)$, the stationarity condition is precisely $DF(S)=E$. ConsequentlyThus the derivative of each approximate energy is its own mean stress, includingFor positive linear elasticity, $W=e:C(x)e/2$ and $W^{\ast }=S:C(x{)}^{-1}S/2$, giving the useful check $C_V=⟨C⟩$ and $C_R=⟨C^{-1}{⟩}^{-1}$. More general convex duality gives the same conjugacy relation with subgradients when differentiability or strict convexity is absent.

Let $C=\mathrm{conv}\{a,b\}$. Maximizing a linear functional over this line segment occurs at an endpoint, soThis identifies $f$ as a support function. Its convex conjugate is the indicator functional ${\delta }_C$: if $p\in C$, then $⟨p,x⟩-{\sigma }_C(x)\le 0$ for every $x$, with equality at zero; if $p\notin C$, strict separation and positive scaling of the separating vector make the supremum infinite.

For a convex function, equality in the Fenchel–Young inequality characterizes its subdifferential. HenceThis is support-function subgradients as exposed faces. In particular a maximizing $p$ really is a subgradient, since ${\sigma }_C(z)\ge ⟨p,z⟩=f(x)+⟨p,z-x⟩$ for every $z$.

Writing $p=ta+(1-t)b$, with $0\le t\le 1$, now givesIf $a=b$, the last line is the singleton $\{a\}$, so the formula includes that case.

Compute the convex conjugate of $f_z$ at a general pair $(v,y)$. With $w=z+u-x$, the independent variables become $(x,w)$ and $u=w+x-z$, soTherefore the Lagrange dual function isUsing strong duality from the preceding solution givesEquivalently the conjugate of an infimal convolution is $F^{\ast }=k^{\ast }+h^{\ast }$, and the continuous convex $F$ equals its biconjugate.

For practical subgradient computation, minimize the known convex dual objectiveEvery optimizer, and only an optimizer, belongs to $\partial F(z)$ by Fenchel–Young inequality. The full characterization isThis is infimal-convolution dual subgradients; the set is nonempty because $F$ is finite convex everywhere. If both conjugates are differentiable at the optimizer, solve $\nabla k^{\ast }(y)+\nabla h^{\ast }(y)=z$. For nonsmooth conjugates, use the displayed aggregate subdifferential or a convex minimization algorithm. Replacing it by $\partial k^{\ast }(y)+\partial h^{\ast }(y)$ requires the usual subdifferential sum rule qualification; it is not automatically justified solely by knowing the two conjugates.

LetThe proximal operator optimality condition saysBy subgradient inversion under convex conjugacy, which uses Fenchel–Young inequality and $f^{\ast \ast }=f$ from the Fenchel-Moreau theorem,Consequently $z-p=\tau v\in \tau \partial f^{\ast }(p)$, meaning $p=B_{\tau f^{\ast }}(z)$. Uniqueness of the backward subgradient step identifies the result:This is the scaled Moreau decomposition. It requires one backward step on $f$, with reciprocal parameter $1/\tau$ and scaled input $z/\tau$, followed by a scalar multiplication and subtraction. No separate proximal computation of the convex conjugate is needed.

The convex Legendre transform, or convex conjugate, isFor a differentiable strictly convex function, an attained maximum is characterized by $p=\nabla f(x)$, and then $f^{\ast }(p)=p^{T}x-f(x)$ at that maximizing $x$.

For the affine covariance of the convex conjugate, assume $\lambda >0$ and substitute $y=x-x_0$:HencePositive scaling is necessary for this supremum definition, although the printed formula does not specify it. For example, $f(x)=x^2/2$ and $\lambda =-1$ give $g^{\ast }(p)=+\infty$, not the finite expression $-p^2/2$. If the transform is instead defined as a stationary value using a locally invertible gradient, the same algebraic identity holds for any $\lambda \ne 0$: the critical-point relation is $p=\lambda \nabla f(y)$, and substitution of that stationary $y$ gives the displayed formula. That extension does not assert a supremum for negative scaling.

For the positive-definite quadratic form, symmetry and positive eigenvalues of $A$ giveThe second term is nonpositive and vanishes exactly at $x=A^{-1}p$. Therefore