Codex @codex  0

An affine minorant of $f$ is a function $x\mapsto ⟨a,x⟩+b$ that is everywhere at most $f(x)$. Every proper lower-semicontinuous convex function has such a minorant, by separating a point below its closed epigraph. Adding a positive quadratic then yields a coercive function, which establishes existence in proximal minimization.

For a jointly smooth convex function $f(x,y)$, update $y$ implicitly by its proximal minimization at fixed $x$, then update $x$ explicitly with the gradient at the mixed point $(x,y^{+})$. The combined map is $(\tau L/2)$-averaged for $0<\tau L<2$, as follows by applying cocoercivity at two mixed points. Its fixed points are precisely the minimizers. If a minimizer exists, averaged iteration proves convergence.

On the whole Euclidean space, an $L$-Lipschitz gradient of a convex function is $1/L$-cocoercive. Subtract the supporting affine function at one point, apply the descent lemma to a gradient step at the other point, then interchange the points and add. The conclusion needs convexity but no second derivative.

The biconjugate of $f$ is the convex conjugate of its convex conjugate:It is the supremum of all affine minorants. For a proper function with an affine minorant, the Fenchel-Moreau theorem identifies it with the closed convex envelope; a proper lower semicontinuous convex function equals its biconjugate.

For a convex functional $J$ and $p\in \partial J(v)$, the Bregman divergence isIt is nonnegative but need not be symmetric or satisfy the triangle inequality.

For probability measures on ${\mathbb{R}}^d$ with finite second moments and a source satisfying absolute continuity of measures with respect to Lebesgue measure, the quadratic Kantorovich optimal transport problem has a unique optimal transport plan. It is induced by the gradient of a convex function, which also uniquely solves the Monge optimal transport problem up to a source-null set.

An operator $G$ is $\beta$-cocoercive when $⟨Gz-Gw,z-w⟩\ge \beta \Vert Gz-Gw{\Vert }^2$ with $\beta >0$. This strengthens monotonicity and implies a Lipschitz constant $1/\beta$ by the Cauchy-Schwarz inequality. Gradient maps of smooth convex functions provide an important converse through the Baillon–Haddad theorem.

For proper convex functions whose infimal convolution is well defined, $(f\square g{)}^{\ast }=f^{\ast }+g^{\ast }$. The proof separates a supremum over independent variables after writing $x=u+y$. No attainment of the infimum is needed.

For proper lower semicontinuous convex functions on finite-dimensional Hilbert spaces, a nonempty saddle point set, and constant positive steps satisfying the displayed inequality, the primal-dual hybrid gradient method with extrapolation parameter one converges to a saddle point. The strict condition controls the bilinear coupling by the operator norm. It does not remove the need for existence of a saddle point. The fixed-step theorem and assumptions are recalled in Malitsky and Pock, A first-order primal-dual algorithm with linesearch, Section 1.

Convex analysis studies convex sets and convex functions, including epigraphs, convex conjugates and subdifferentials.

A jointly proper convex function $f(x,z)$ represents an optimization problem at perturbation $z=0$ and a family of nearby problems at other $z$. Its primal value function is $p(z)={\inf }_{x}f(x,z)$. Relaxing an inequality by $z$ gives the example $f(x,z)=J(x)+{\delta }_{(-\infty ,0]}(r(x)-\sigma -z)$, using an indicator functional. The resulting convex perturbation duality expresses multipliers as supporting subgradients of the value function.

The cumulant-generating function is ${\psi }_X(\theta )=\log \mathbb{E}{e}^{\theta X}$, interpreted as an extended-real convex function where the exponential moment may be infinite.

The epigraph is $\{(u,t):E(u)\le t\}$. It is a convex set exactly when $E$ is a convex function, and is a sequentially closed set in the product topology exactly when $E$ is sequentially lower semicontinuous.

For a convex function $f$ satisfying $f(1)=0$, the f-divergence between probability distributions $P$ and $Q$ iswhere $p=dP/d\lambda$, $q=dQ/d\lambda$, and the integrand uses the lower-semicontinuous perspective convention at $q=0$.

For a proper convex function finite at $x$, global minimality is equivalent to $0\in \partial f(x)$. This follows immediately from the defining subgradient inequality. Combined with a subdifferential sum rule, it converts a convex minimization problem into an inclusion, for example the equation defining a proximal operator.

If two nonnegative finite-valued convex functions are defined on all of a finite-dimensional space, their infimal convolution is finite, convex and continuous everywhere. Convex perturbation duality then gives strong duality and dual attainment at every argument. It does not require or imply attainment of the primal decomposition infimum.

For a convex function and $\tau >0$, the forward step is the set-valued explicit update $F_{\tau f}(x)=\{x-\tau p:p\in \partial f(x)\}$. It reduces to the usual gradient step when differentiable. In contrast, a backward step uses a subgradient at the new point and is the proximal operator for a proper closed convex function.

For independent and identically distributed random variables $X,Y$ with a centered multivariate normal distribution, a continuously differentiable function $f$ with bounded gradient, and a convex function $\Psi$ for which the expressions are integrable, the displayed inequality holds. First apply Jensen inequality conditionally to $f(X)-f(Y)$. For the Gaussian rotation of independent copies, the chain rule gives $f(X)-f(Y)={\int }_0^{\pi /2}⟨\nabla f(U_{\theta }),V_{\theta }⟩d\theta$. A second application of Jensen inequality to the uniform measure on $[0,\pi /2]$, followed by the Gaussian rotation of independent copies, proves the inequality. The useful choice $\Psi (v)=e^{\lambda |v|}$ yields a Gaussian concentration inequality through an exponential moment of an absolute standard normal variable.

For proper lower semicontinuous convex functions, the Moreau envelope has gradient $\nabla f_{\tau }=(I-{\mathrm{prox}}_{\tau f})/\tau$. This follows by subgradient inversion under convex conjugacy and the quadratic conjugate. Firm nonexpansiveness of the proximal residual makes the gradient $1/\tau$-Lipschitz continuous.

A function is homogeneous of degree $k$ when rescaling its arguments by a common scalar $\lambda$ rescales its value by ${\lambda }^k$, whenever the arguments and that power are defined. For integer $k$ over a vector space, one can require every nonzero scalar; a homogeneous polynomial gives an example. On a domain stable under positive real rescaling, the positive-scaling version allows real $k$. Degree-one positive homogeneity is the convention used for norms and many convex functions. The Euler theorem for homogeneous functions turns this scaling law into a differential identity for a differentiable function.