= Legendre transform of a cumulant-generating function
{title2=$\psi^*$}
The <Legendre transform of a cumulant-generating function> $K(\theta)=\log\mathbb E e^{\theta X}$ is
$$
I(x)=K^*(x)=\sup_{\theta\in\mathbb R}\{\theta x-K(\theta)\},
$$
where parameters with $K(\theta)=+\infty$ contribute nothing to the supremum. More generally this is the <Legendre-Fenchel transform>. It is convex because it is a supremum of <affine function>[affine functions] of $x$, and is nonnegative because $K(0)=0$ allows the choice $\theta=0$.
If $K$ is differentiable and the supremum is attained at an interior parameter, its maximizer solves $K'(\theta)=x$. For a <Normal distribution> with <mean> $\mu$ and <variance> $\sigma^2>0$, $K(\theta)=\mu\theta+\sigma^2\theta^2/2$, so $\theta=(x-\mu)/\sigma^2$ and $I(x)=(x-\mu)^2/(2\sigma^2)$. The zero cost occurs at the <mean> and the cost grows quadratically away from it.
For <independent and identically distributed>[iid] sample <means>, <Cramér's theorem> identifies this transform as the large-deviation <Rate function> when the <Moment-generating function> is finite on a neighbourhood of zero. The optimization also appears directly in a <Chernoff bound>: optimize the <Markov inequality>[exponential Markov inequality] over nonnegative parameters to bound an upper-tail <probability>. The transform converts exponential growth of a generating <function> into exponential decay of a rare-event <probability>.
Back to article page
