Convex function ($f(tx+(1-t)y)\le tf(x)+(1-t)f(y)$)

A function $f:C\to \mathbb{R}$ on a convex set $C$ is a convex function if, for every $x,y\in C$ and $t\in [0,1]$,Geometrically, a line segment joining two points of its graph of a function lies on or above that graph of a function. The convex set condition ensures that the intermediate input stays in the domain of a function. An affine function satisfies the defining inequality with equality.

For the quadratic function $f(x)=x^2$, the inputs $1$ and $3$ have equally weighted arithmetic mean $2$. Evaluating at the arithmetic mean gives $f(2)=4$, whereas averaging the outputs gives $(f(1)+f(3))/2=5$. Averaging first saves one unit in this example. More generally, direct expansion givesThis identity proves convexity and also shows exactly when its defining inequality is strict.

For a differentiable convex function, the tangent line in one dimension, or the corresponding supporting affine function in several dimensions, lies below the entire graph of a function. This is the subgradient inequality. The equivalent geometric statement is that the epigraph, consisting of points on or above the graph of a function, is a convex set. These characterizations are developed in Boyd and Vandenberghe's convex-function slides, sections 3.1–3.4.

Convexity does not require differentiability: the absolute value function $|x|$ is a convex function with a corner at $0$. When a function is twice continuously differentiable on an open interval, it is a convex function exactly when its second derivative is nonnegative throughout that interval. In several dimensions, the corresponding criterion on an open convex set is that its Hessian matrix is positive semidefinite everywhere. The domain of a function matters: $x^3$ is a convex function on $[0,\infty )$, but not on all of $\mathbb{R}$. A strictly positive second derivative is sufficient for strict convexity, but is not necessary: $x^4$ is a strictly convex function although its second derivative vanishes at $0$.

The usefulness of convexity is that local information can certify a global conclusion. Every local minimum of a convex function on a convex set is a global minimum: if a lower value existed elsewhere, the defining inequality would produce lower values arbitrarily close to the proposed local minimum. For a differentiable convex function on an open convex set, a zero gradient therefore certifies a global minimum. Strict convexity gives uniqueness when a global minimum exists; it does not supply existence by itself. These facts explain the role of convex functions in convex optimization.

For a differentiable real function $g$, a point of convexity is a point $v_0$ whose tangent line lies below the entire graph. The function need not be convex at other points. The Weierstrass excess function is exactly the difference between the function and this tangent line when applied to a variational integrand as a function of slope.

On an open convex set, a twice-differentiable real-valued function is convex if and only if its Hessian matrix is a positive semidefinite matrix at every point. In particular continuous second partial derivatives suffice for the differentiability hypothesis. Restrict $g$ to any line segment: $\varphi (t)=g(x+tv)$ has ${\varphi }^{\prime \prime }(t)=v^{\mathsf{T}}H(g)(x+tv)v$. The second-derivative criterion for convexity proves both implications, because convexity is equivalent to convexity of all such restrictions.

A twice-differentiable real-valued function on an open interval is convex if and only if its second derivative is nonnegative everywhere. If $f^{\prime \prime }\ge 0$, the mean value theorem makes $f^{\prime }$ nondecreasing, which implies the supporting-line and chord inequalities for convexity. Conversely, the slopes of a convex function are ordered; where the second derivative exists, this gives $f^{\prime \prime }\ge 0$. Strict positivity of $f^{\prime \prime }$ is sufficient for strict convexity, but is not necessary: $f(x)=x^4$ is strictly convex despite $f^{\prime \prime }(0)=0$.

The affine functions $x$ and $-x$ are both convex functions on $\mathbb{R}$, but their product is $-x^2$. Its value at zero exceeds the average of its values at $-1$ and $1$, contradicting convexity. Thus closure under nonnegative linear combinations does not imply closure under products.

For a differentiable real-valued function on an open convex set, convexity is equivalent to the displayed inequality for all $x,y$ in its domain. To prove the forward direction, convexity gives $[f(x+t(y-x))-f(x)]/t\le f(y)-f(x)$ for $0<t\le 1$. Take the limit as $t\downarrow 0$ using first-order Taylor expansion.

Conversely, put $z=(1-t)x+ty$ and apply the supporting inequality at $z$ to both $x$ and $y$. Multiply the two inequalities by $1-t$ and $t$ and add them. The gradient terms cancel because $(1-t)(x-z)+t(y-z)=0$, leaving $f(z)\le (1-t)f(x)+tf(y)$. This is the defining convexity inequality. The endpoint values $t=0,1$ hold as equalities.

Let $f$ be a finite convex function on a neighbourhood of $x\in {\mathbb{R}}^d$, and suppose its first partial derivatives exist at $x$. For any fixed vector $h$, convexity givesThe equality uses the one-dimensional first-order Taylor expansion along each coordinate line; no continuity of second partial derivatives is assumed. Applying this upper bound to $-h$ and using $f(x)\le [f(x+th)+f(x-th)]/2$ gives the matching lower bound. Hence the displayed directional expansion holds. This supplies the expansion used in the supporting inequality for a differentiable convex function even when continuous second partial derivatives have not been postulated.

For a continuous convex function on $[a,b]$, where $a<b$, its integral average lies between its midpoint value and the average of its endpoint values. For a concave function both inequalities reverse. The bound involving endpoints follows by comparison with the affine function joining them and monotonicity of the Riemann integral. The midpoint bound follows from the Jensen inequality on equal-weight midpoint tags and the Riemann sum convergence theorem.

A functional on a real vector space is convex when it is a convex function: $J(\theta v+(1-\theta )w)\le \theta J(v)+(1-\theta )J(w)$ for $0\le \theta \le 1$. For a symmetric nonnegative bilinear form $a$ and a linear functional $\ell$, the quadratic functional $J(v)=\frac{1}{2}a(v,v)-\ell (v)$ is convex, since its convexity difference is $\frac{1}{2}\theta (1-\theta )a(v-w,v-w)\ge 0$.

The greatest convex function lying below a given function, when such an envelope is considered in the specified domain. A convex combination of phase compositions and their Gibbs free energies constructs candidate mixtures; the equilibrium free energy is their lower convex envelope. For two differentiable coexisting branches, the supporting line is a common tangent.

For a finite real convex function on an entire real vector space, apply the convex domination form of the Hahn-Banach theorem to $\varphi (y+z)-\varphi (y)$ and the zero functional on $\{0\}$. It supplies a linear functional ${\ell }_y$ dominated by this translated function. The affine function $a_y(x)=\varphi (y)+{\ell }_y(x-y)$ lies below $\varphi$ everywhere and equals it at $y$. Taking the supremum proves the formula. These are algebraic affine minorants; continuity requires additional topological hypotheses.

An extended-real convex function $\Lambda :{\mathbb{R}}^d\to \mathbb{R}\cup \{+\infty \}$ is essentially smooth if its effective domain has nonempty interior, it is differentiable throughout that interior, and $\Vert \nabla \Lambda ({\theta }_n)\Vert \to \infty$ whenever interior points converge to a finite boundary point of the domain. A differentiable convex function finite on all of ${\mathbb{R}}^d$ satisfies the boundary condition vacuously. The cumulant-generating function $\log (r/(r-\theta ))$ on $\theta <r$ has derivative $(r-\theta {)}^{-1}$ and is essentially smooth. Restricting its domain to $\theta <\lambda <r$ destroys this property. This distinction matters in the lower bounds of the Gärtner–Ellis theorem.

A convex quadratic function on ${\mathbb{R}}^n$ has the form $q(x)=\frac{1}{2}{x}^{T}Hx+b^{T}x+c$ with a symmetric positive semidefinite matrix $H$. Its Hessian is $H$, and $q(y)-q(x)-\nabla q(x)\cdot (y-x)=\frac{1}{2}(y-x{)}^{T}H(y-x)\ge 0$, proving convexity directly. A stationary point is a global minimum. On a compact polytope, a minimum either is stationary in the interior or minimizes a restriction to a boundary face.

Let a real convex function satisfy $|f|\le M$ on the open unit ball of a normed space. For $4r=1-\Vert x\Vert$, points $u,v\in B(x,r)$ satisfy $|f(u)-f(v)|\le (M/r)\Vert u-v\Vert$. Extend the segment from $u$ through $v$ to length $2r$; its endpoint still lies in the domain, and convexity bounds the difference by $2M\Vert u-v\Vert /(2r)$. Reverse $u,v$ for the opposite inequality. Unlike finite-dimensional local continuity theorems, this argument applies in arbitrary dimension because boundedness on a full neighborhood is assumed.

For a finite convex function $f:\mathbb{R}\to \mathbb{R}$, its negative part $f^{-}=\max (-f,0)$ is globally Lipschitz continuous. The negative sublevel set is an interval. If bounded, local Lipschitz continuity of finite convex functions bounds the slopes on its closure. If it is a left ray, any negative slope there would force $f$ positive far to the left by the supporting-line inequality; all slopes on that ray are therefore nonnegative and bounded above near its finite endpoint. For a right ray the analogous slopes are nonpositive and bounded below. If $f$ is negative everywhere, convexity and boundedness above force it to be constant. These cases bound the slopes of $f^{-}$ globally, including its zero extension outside the interval. In particular, integrability of $f(Z)$ implies integrability of the negative part of every fixed translate $f(Z+x)$.

An affine minorant of $f$ is a function $x\mapsto ⟨a,x⟩+b$ that is everywhere at most $f(x)$. Every proper lower-semicontinuous convex function has such a minorant, by separating a point below its closed epigraph. Adding a positive quadratic then yields a coercive function, which establishes existence in proximal minimization.

A proper convex function is both a convex function and a proper extended-real function. For example, the total variation seminorm on a domain is nonnegative, finite at zero, and a supremum of linear functionals, making it proper and convex.

A real function $f$ on an interval $I$ is operator convex whenin the Loewner order for all Hermitian operators $A,B$ with spectra in $I$ and every $0\le \lambda \le 1$.

A function is operator concave when its negative is operator convex. Equivalently, the defining operator-convexity inequality is reversed.

A finite convex function on an open convex subset of a finite-dimensional real vector space is locally Lipschitz, and therefore continuous. In one dimension this follows by bounding its secant slopes between two slightly wider endpoints.

The pointwise maximum of finitely many convex functions is convex because

The softplus function $s(x)=\log (1+e^x)$ is convex because

The absolute value function is convex by the triangle inequality:

For a real number $x$, its absolute value is $x$ when $x\ge 0$ and $-x$ otherwise; for a complex number $z$, it is the nonnegative magnitude $\sqrt{z\overline{z}}$.

Suppose a real function $g$ is differentiable at $a$, with $g(a)=0$ and $g^{\prime }(a)\ne 0$. The derivative definition gives $g(a+h)=g^{\prime }(a)h+o(|h|)$. The reverse triangle inequality then impliesThus $|g|$ has unequal one-sided derivatives, $|g^{\prime }(a)|$ on the right and $-|g^{\prime }(a)|$ on the left. More generally, multiplication by a function $w$ continuous at $a$ with $w(a)\ne 0$ gives right and left slopes $w(a)|g^{\prime }(a)|$ and $-w(a)|g^{\prime }(a)|$. This elementary expansion explains how a smooth oscillation can acquire infinitely many corners under the absolute value function, even after multiplication by a factor that makes it differentiable at an accumulation point.

Let the locations $a_n$ be distinct and bounded, and let $c_n>0$ with ${\sum }_n{c}_n<\infty$. The Weierstrass M-test gives locally uniform convergence of $G$, so the uniform limit theorem makes it continuous. The reverse triangle inequality bounds each term's difference quotient by $c_n$. Consequently a uniformly small tail permits passing each one-sided difference-quotient limit through the series. At every point outside the locations, $G^{\prime }(x)={\sum }_n{c}_n\mathrm{sgn}(x-a_n)$; at $a_m$, the right derivative minus the left derivative is $2c_m>0$. Thus the function has precisely the prescribed cusp locations, even when they accumulate at another point where the function remains differentiable.

A differentiable function is $\alpha$-strongly convex when

For a differentiable strongly convex function with modulus $\gamma >0$ and global minimizer $x^{\ast }$, the supporting inequality givesCompleting the square shows that the last two terms are at least $-\Vert \nabla f(x){\Vert }^2/(2\gamma )$, which proves the result. Combining this with the descent lemma converts squared gradient decrease into a contraction of the objective function gap.

Target: mathematical-optimization.bigb, under Line search.

A gradient integrand is uniformly convex in this elliptic variational sense when its Hessian matrix in the gradient variables has a common positive lower bound. Then $(F_p(x,p)-F_p(x,q))\cdot (p-q)\ge \lambda |p-q{|}^2$. Averaging $F_{pp}$ along a line segment in gradient space preserves the same lower bound.

For scalar minimizers of $\int F(Du)$ with $F\in C^2({\mathbb{R}}^n)$ and globally bounded uniformly positive Hessian matrix, no higher differentiability of $F$ is needed for interior gradient Hölder continuity. The weak Euler-Lagrange equation is $\mathrm{div}DF(Du)=0$. Difference quotients solve uniformly elliptic equations with coefficient matrices obtained by averaging $D^{2}F$ along gradient segments. The Caccioppoli inequality and difference quotient characterization give $u\in H_{\mathrm{loc}}^2$. The Sobolev chain rule then gives $\mathrm{div}(D^{2}F(Du)D(D_{k}u))=0$. The De Giorgi-Nash-Moser theorem makes each derivative Hölder continuous, with exponent depending only on dimension and the ellipticity ratio. This scalar conclusion should not be assumed for vector-valued elliptic systems.

On a bounded domain, classical solutions continuous on the closure have difference $w=u-v$ satisfying $D_i(A_{ij}{D}_{j}w)=0$, where $A={\int }_0^1{F}_{pp}(x,Dv+tDw)dt$. The coefficients are uniformly elliptic and locally smooth. Apply the weak maximum principle on inner domains whose closures lie in the original domain, then let their boundary distances tend to zero. The common continuous boundary values make their boundary differences tend uniformly to zero, proving uniqueness without assuming global bounds on derivatives of the linearized coefficients.

A function $f$ on a convex set is strictly convex whenwhenever $x\ne y$ and $0<t<1$.

A strictly convex function has at most one minimizer: if distinct points attained the same minimum, every strict convex combination of them would have a smaller value.

The Hessian matrix ofis positive semidefinite exactly whenThis region is convex: for $A\ne 0$ it is the epigraph of $A^2/(36x)$ in the positive quadrant, and for $A=0$ it is the closed first quadrant.

The perspective of f is g(t,x)=t f(x/t) for positive t and preserves convexity.

A subgradient of a convex function $f$ at $x$ is a vector $g$ such that $f(y)\ge f(x)+g^T(y-x)$ for every $y$ in the domain.

A real convex function locally bounded on an open convex set has a continuous supporting linear functional at every interior point. Apply the dominated Hahn-Banach theorem to its sublinear directional derivative $p_x$ to obtain $l\le p_x$. Increasing secant slopes imply $p_x(z)\le f(x+z)-f(x)$ whenever the endpoint is in the domain. A local Lipschitz continuity bound gives both $l(y)\le L\Vert y\Vert$ and $-l(y)=l(-y)\le L\Vert y\Vert$, proving continuity.

A proper convex function $f$ is minimized at $x$ exactly when $0\in \partial f(x)$. For a differentiable convex term $g$ and another convex term $h$, the subdifferential sum rule gives the condition $0\in \nabla g(x)+\partial h(x)$.

The absolute value function has

A vector is a subgradient when its supporting affine function lies below the convex function.

Jensen's inequality says that applying a convex function to a weighted mean cannot exceed the same weighted mean of its outputs. For nonnegative weights $w_i$ with ${\sum }_{i=1}^n{w}_i=1$ and inputs $x_i$ in a convex set,Nonnegative weights are essential: this is a convex combination, rather than an arbitrary affine combination. For two inputs it is precisely the defining inequality for a convex function; repeatedly grouping inputs gives the finite weighted version.

Take $f(x)=x^2$, weights $1/4$ and $3/4$, and inputs $1$ and $3$. Their weighted mean is $2.5$, so the two sides areThe Jensen gap is $0.75$. The inequality measures the effect of replacing distinct inputs by one representative input before applying a nonlinear function.

The probabilistic form uses an integrable random variable $X$ taking values in an interval where $f$ is a convex function. If $f(X)$ is also an integrable random variable and $\mathbb{E}X$ belongs to that interval, thenThe direction reverses for a concave function. For a strictly convex function, equality in the finite weighted form requires all inputs having positive weight to be equal; in the probabilistic form it requires $X$ to be constant almost surely. An affine function gives equality for any admissible distribution. ^[ref].

For the quadratic function $f(x)=x^2$ and a random variable with a finite second moment, the difference between the two sides is exactly its variance:Consider two equally likely values $2-d$ and $2+d$. Their expected value stays at $2$ as $d$ increases, but their variance is $d^2$. Thus the expected value of the square grows from $4$ to $4+d^2$. For the exponential function, the same two-point distribution gives $\mathbb{E}[e^X]=e^2\cosh d$, while $e^{\mathbb{E}X}=e^2$. Both are examples of Jensen's inequality, but their Jensen gaps depend differently on the spread of the inputs.

Another useful choice is the concave function $\log x$ on positive inputs. Its version of Jensen's inequality gives ${\sum }_i{w}_i\log x_i\le \log ({\sum }_i{w}_i{x}_i)$. Applying the increasing exponential function shows that the weighted geometric mean cannot exceed the weighted arithmetic mean. For the equally weighted inputs $1$ and $9$, these are $3$ and $5$. This recovers the arithmetic-geometric mean inequality and illustrates how choosing one function can generate an entire family of bounds.

A short proof idea explains the probabilistic form. If $f$ has a subgradient $g$ at $\mu =\mathbb{E}X$, the supporting affine function gives $f(X)\ge f(\mu )+g(X-\mu )$. Taking the expected value cancels the linear term because $\mathbb{E}[X-\mu ]=0$. This argument applies, for example, to finite convex functions on $\mathbb{R}$; more general domains of a function need attention to endpoints and the existence of a subgradient.

Johan Jensen's 1906 paper “Sur les fonctions convexes et les inégalités entre les valeurs moyennes” is an early source for the connection between convex functions and inequalities for means. ^[ref].

Let $X$ be an integrable random variable taking values in an open interval $I$. Suppose $f:I\to \mathbb{R}$ is a differentiable strictly convex function. Then its expected value has equality in Jensen's inequality exactly when $X$ is constant almost surely.

First $m=\mathbb{E}X$ lies in $I$: at each finite endpoint, taking the expected value of the strictly positive distance to that endpoint gives a strictly positive distance for $m$. A tangent affine function supports $f$ at $m$, soStrict convexity forces $h(t)>0$ for $t\ne m$. Indeed, equality at another point would force the graph to coincide with its supporting line on the connecting line segment, contradicting strict convexity. Thereforeand the right side is zero exactly when $X=m$ almost surely.

The tangent affine function of $X$ is an integrable random variable, so $f(X)$ has an integrable negative part. Thus $\mathbb{E}[f(X)]$ is well defined, possibly $+\infty$, even if absolute integrability was not assumed. Equality to $f(m)$ automatically excludes that infinite case.

The Jensen gap is the nonnegative difference $\mathbb{E}[f(X)]-f(\mathbb{E}X)$ in Jensen's inequality, under its integrability assumptions. It quantifies how much applying a convex function before averaging differs from applying it afterward. For $f(x)=x^2$ it is the variance of $X$; for an affine function it is zero. For general convex functions, it depends on both the function and the distribution of $X$, and is not determined by the variance alone.

Apply Jensen's inequality to the two equally weighted values $a,b$ and the convex function $t\mapsto t^p$. Equality for $a=b>0$ proves that $2^{p-1}$ is the smallest possible constant. For $p>1$, strict convexity makes equality equivalent to $a=b$; for $p=1$, equality holds for every pair.