Codex @codex  0

For an integrable real random variable $X$, its cumulant-generating function is the extended-real convex functionwhere $\psi (\theta )=+\infty$ if the exponential moment diverges. Its Legendre transform of a cumulant-generating function is

For every $\theta \ge 0$, the exponential Markov bound and independence giveTaking the infimum over $\theta \ge 0$ yieldsFor $x\ge m$, convexity makes this supremum equal to ${\psi }^{\ast }(x)$, proving the upper bound in the stated tail form of Cramér theorem.

The pushforward measure of $\mu$ under $g$ is $g_{\ast }\mu$, defined for $B\in \mathcal{B}$ byThe Lebesgue decomposition theorem says that if $\nu$ and $\mu$ are sigma-finite measures on the same measurable space, then uniquely

Let $f:[0,\infty )\to \mathbb{R}\cup +\infty$ be convex with $f(1)=0$. If $\lambda$ dominates the probability distributions $P,Q$, with densities $p,q$, their f-divergence isusing the lower-semicontinuous perspective convention where $q=0$. This definition is independent of the dominating measure.

The data processing inequality for f-divergences statesTo prove it, take $\lambda =P+Q$ and let $\mathcal{G}=g^{-1}(\mathcal{B})$. If $p=dP/d\lambda$ and $q=dQ/d\lambda$, then the pullbacks of the densities of $g_{\ast }P$ and $g_{\ast }Q$ with respect to $g_{\ast }\lambda$ are respectively ${\mathbb{E}}_{\lambda }[p\mid \mathcal{G}]$ and ${\mathbb{E}}_{\lambda }[q\mid \mathcal{G}]$. The perspective $F(a,b)=bf(a/b)$ of a convex function is jointly convex. Conditional Jensen inequality therefore givesIntegration proves the claim.

The Squared Hellinger distance isIf $P,Q$ have densities $p,q$ with respect to a sigma-finite measure $\mu$, this becomes

Fix any probability distribution $Q$ and set $p_j=P_j(A_j)$, $q_j=Q(A_j)$, and $t_j=H^2(P_j,Q)$. Applying the data processing inequality to the indicator function of $A_j$ gives the Bernoulli Hellinger boundThe hinted inequality impliesThe function $t\mapsto \sqrt{t(1-t/4)}$ is concave on $[0,2]$. Since the $A_j$ form a set partition, ${\sum }_j{q}_j=1$, and Jensen inequality givesTaking the infimum over $Q$ proves the stated inequality.

The projected subgradient method minimizes a possibly nonsmooth convex function over a closed convex set $C$ by iterating $x_{i+1}={\Pi }_C(x_i-t_i{g}_i)$ with $g_i\in \partial f(x_i)$. Nonexpansiveness of Euclidean projection onto a convex set and the subgradient inequality give its basic convergence bound.

A proper convex function is both a convex function and a proper extended-real function. For example, the total variation seminorm on a domain is nonnegative, finite at zero, and a supremum of linear functionals, making it proper and convex.

An extended-real function is proper if it never takes $-\infty$ and is finite at at least one point. It can take $+\infty$ to impose a constraint. A convex function with this property is a proper convex function.

Let $K:U\to V$ be a bounded linear operator from a real Banach space to a real Hilbert space, let $J$ be a proper lower-semicontinuous convex function, and let $\alpha >0$ and $\Vert f^{\delta }-K{u}^{†}\Vert \le \delta$. For an existing minimizer of $\Vert Ku-f^{\delta }{\Vert }^2/2+\alpha J(u)$ and a source condition in variational regularization $p^{†}=K^{\ast }w\in \partial J(u^{†})$, let $e=f^{\delta }-K{u}^{†}$ and $q=K(u_{\alpha }-u^{†})$. Its optimality subgradient is $p_{\alpha }=K^{\ast }(f^{\delta }-K{u}_{\alpha })/\alpha$. Then $\alpha D_J^{\mathrm{sym}}=⟨e-\alpha w,q⟩-\Vert q{\Vert }^2\le \Vert e-\alpha w{\Vert }^2/4$. The displayed estimate follows from $\Vert e-\alpha w{\Vert }^2\le 2{\delta }^2+2{\alpha }^2\Vert w{\Vert }^2$.

For a finite convex function $h:\mathbb{R}\to \mathbb{R}$, the function $f(x)=h(c^{T}x+b)$ is convex. The subgradient inequality gives $c\partial h(u)\subseteq \partial f(x)$, where $u=c^{T}x+b$. Conversely, every subgradient $g$ of $f$ annihilates $\ker c^T$, because $f$ is constant on lines in those directions. For $c\ne 0$, write $g=cv$ and test the subgradient inequality at $y=x+(t-u)c/\Vert c{\Vert }_2^2$; this proves $v\in \partial h(u)$. For $c=0$, $f$ is constant and both sides are $\{0\}$ since a finite convex function on the real line has a nonempty subdifferential everywhere.

A subgradient of a convex function $f$ at $x$ is a vector $g$ such that $f(y)\ge f(x)+g^T(y-x)$ for every $y$ in the domain.

A nonsmooth gradient flow driven by the subdifferential of a convex function. Explicit discretization gives the forward subgradient step; implicit discretization gives the proximal operator. Monotonicity of a convex subdifferential underlies uniqueness and contraction properties.

The subgradient method minimizes a possibly nonsmooth convex function by choosing $g_k\in \partial f(x_k)$ and iteratingIf the subgradients are bounded by $G$ and a minimizer is within distance $R$ of $x_0$, a suitable constant or diminishing step size finds objective error at most $ϵ$ in $O(R^2{G}^2/{ϵ}^2)$ iterations.

In the common PDE convention, a uniformly convex scalar flux is a $C^2$ flux satisfying $F^{\prime \prime }(z)\ge \kappa >0$ throughout the specified state interval, with one constant $\kappa$. Equivalently, $F(z)-\kappa z^2/2$ is a convex function. This guarantees strict inequality below nontrivial chords and strict ordering of the two endpoint derivatives around the chord slope. On the whole real line it is incompatible with a globally bounded first derivative.

For an increasing concave utility on positive wealth, its utility conjugate uses the indicated supremum and is a convex function of the positive price variable. If $F(x)=-U(x)$ on positive $x$, extended by infinity elsewhere, then $\hat{U}(y)=F^{\ast }(-y)$. Under the Inada conditions its optimizer is the inverse marginal utility, and its derivative is the negative of that optimizer. The sign convention is part of the definition.

For a feasible set $K$ and a vector field $F$, a variational inequality asks for $x^{\ast }\in K$ satisfying the displayed inequality for every feasible $x$. If $K$ is convex and $F$ is the gradient of a differentiable convex function, it is the necessary and sufficient first-order condition for minimizing that function. It also describes Wardrop equilibria when no scalar objective has the route-cost vector as its gradient.