Jensen's inequality ($f(\mathbb{E}X)\le \mathbb{E}[f(X)]$)

Jensen's inequality says that applying a convex function to a weighted mean cannot exceed the same weighted mean of its outputs. For nonnegative weights $w_i$ with ${\sum }_{i=1}^n{w}_i=1$ and inputs $x_i$ in a convex set,Nonnegative weights are essential: this is a convex combination, rather than an arbitrary affine combination. For two inputs it is precisely the defining inequality for a convex function; repeatedly grouping inputs gives the finite weighted version.

Take $f(x)=x^2$, weights $1/4$ and $3/4$, and inputs $1$ and $3$. Their weighted mean is $2.5$, so the two sides areThe Jensen gap is $0.75$. The inequality measures the effect of replacing distinct inputs by one representative input before applying a nonlinear function.

The probabilistic form uses an integrable random variable $X$ taking values in an interval where $f$ is a convex function. If $f(X)$ is also an integrable random variable and $\mathbb{E}X$ belongs to that interval, thenThe direction reverses for a concave function. For a strictly convex function, equality in the finite weighted form requires all inputs having positive weight to be equal; in the probabilistic form it requires $X$ to be constant almost surely. An affine function gives equality for any admissible distribution. ^[ref].

For the quadratic function $f(x)=x^2$ and a random variable with a finite second moment, the difference between the two sides is exactly its variance:Consider two equally likely values $2-d$ and $2+d$. Their expected value stays at $2$ as $d$ increases, but their variance is $d^2$. Thus the expected value of the square grows from $4$ to $4+d^2$. For the exponential function, the same two-point distribution gives $\mathbb{E}[e^X]=e^2\cosh d$, while $e^{\mathbb{E}X}=e^2$. Both are examples of Jensen's inequality, but their Jensen gaps depend differently on the spread of the inputs.

Another useful choice is the concave function $\log x$ on positive inputs. Its version of Jensen's inequality gives ${\sum }_i{w}_i\log x_i\le \log ({\sum }_i{w}_i{x}_i)$. Applying the increasing exponential function shows that the weighted geometric mean cannot exceed the weighted arithmetic mean. For the equally weighted inputs $1$ and $9$, these are $3$ and $5$. This recovers the arithmetic-geometric mean inequality and illustrates how choosing one function can generate an entire family of bounds.

A short proof idea explains the probabilistic form. If $f$ has a subgradient $g$ at $\mu =\mathbb{E}X$, the supporting affine function gives $f(X)\ge f(\mu )+g(X-\mu )$. Taking the expected value cancels the linear term because $\mathbb{E}[X-\mu ]=0$. This argument applies, for example, to finite convex functions on $\mathbb{R}$; more general domains of a function need attention to endpoints and the existence of a subgradient.

Johan Jensen's 1906 paper “Sur les fonctions convexes et les inégalités entre les valeurs moyennes” is an early source for the connection between convex functions and inequalities for means. ^[ref].

Let $X$ be an integrable random variable taking values in an open interval $I$. Suppose $f:I\to \mathbb{R}$ is a differentiable strictly convex function. Then its expected value has equality in Jensen's inequality exactly when $X$ is constant almost surely.

First $m=\mathbb{E}X$ lies in $I$: at each finite endpoint, taking the expected value of the strictly positive distance to that endpoint gives a strictly positive distance for $m$. A tangent affine function supports $f$ at $m$, soStrict convexity forces $h(t)>0$ for $t\ne m$. Indeed, equality at another point would force the graph to coincide with its supporting line on the connecting line segment, contradicting strict convexity. Thereforeand the right side is zero exactly when $X=m$ almost surely.

The tangent affine function of $X$ is an integrable random variable, so $f(X)$ has an integrable negative part. Thus $\mathbb{E}[f(X)]$ is well defined, possibly $+\infty$, even if absolute integrability was not assumed. Equality to $f(m)$ automatically excludes that infinite case.

The Jensen gap is the nonnegative difference $\mathbb{E}[f(X)]-f(\mathbb{E}X)$ in Jensen's inequality, under its integrability assumptions. It quantifies how much applying a convex function before averaging differs from applying it afterward. For $f(x)=x^2$ it is the variance of $X$; for an affine function it is zero. For general convex functions, it depends on both the function and the distribution of $X$, and is not determined by the variance alone.

Apply Jensen's inequality to the two equally weighted values $a,b$ and the convex function $t\mapsto t^p$. Equality for $a=b>0$ proves that $2^{p-1}$ is the smallest possible constant. For $p>1$, strict convexity makes equality equivalent to $a=b$; for $p=1$, equality holds for every pair.