Cumulant-generating function (${\psi }_X(\theta )$)

The cumulant-generating function is ${\psi }_X(\theta )=\log \mathbb{E}{e}^{\theta X}$, interpreted as an extended-real convex function where the exponential moment may be infinite.

Suppose the moment-generating function $M_X$ is finite on an open interval around zero. Its cumulant-generating function $K_X=\log M_X$ satisfies $K_X(0)=0$, $K_X^{\prime }(0)=\mathbb{E}X=\mu$ and $K_X^{\prime \prime }(0)=\mathrm{Var}(X)={\sigma }^2$. Indeed,and $M_X(0)=1$. The Taylor expansion gives the displayed formula, with little o notation describing the remainder. Finiteness in a neighborhood justifies differentiation: polynomial factors times $e^{tX}$ are dominated on a smaller interval by constants times two slightly larger exponential moments.

For independent identically distributed continuous random variables, let $K(t)=\log \mathbb{E}{e}^{tY}$ be finite near zero. At a mean value $x$ reached by $K^{\prime }(\hat{t})=x$ in the interior of that domain, the displayed leading approximation describes the density of the sum $S_n$. Exponential tilting by $\hat{t}$ puts the sum at its tilted mean; a local normal approximation gives the denominator, and undoing the tilt gives the exponential factor. Under suitable smooth nonlattice regularity, at fixed interior $x$ the exact density is the displayed quantity times $1+O(n^{-1})$. This statistical approximation is an application of the saddle-point approximation to a transform inversion integral. It requires more than the existence of a few ordinary moments.

For independent unit-scale Laplace distributions, the cumulant-generating function is $K(t)=-\log (1-t^2)$ for $|t|<1$. The saddlepoint density approximation solves $K^{\prime }(t)=y$, giving $t=y/(1+\sqrt{1+y^2})$. At this tilt, $K^{\prime \prime }(t)=D(1+D)$ and $ty=D-1$, giving the displayed probability density function approximation for the sample mean. Its value at zero is $\sqrt{n/(4\pi )}$; it need not integrate to one before normalization.

For a sample mean from an independent identically distributed sample, let $K(t)$ be the cumulant-generating function of one observation and solve $K^{\prime }(t)=x$. Put $w=\mathrm{sgn}(t)\sqrt{2n(tx-K(t))}$ and $u=t\sqrt{n{K}^{\prime \prime }(t)}$. Under the smooth, nonlattice conditions for a saddlepoint density approximation, the corresponding approximation to the cumulative distribution function iswhere $\Phi$ and $\varphi$ are the standard normal distribution functions. The apparent singularity at $t=0$ is removable. Write $v=K^{\prime \prime }(0)$ and $k=K^{\prime \prime \prime }(0)$; Taylor expansion gives $w=t\sqrt{nv}(1+kt/(3v)+O(t^2))$ and $u=t\sqrt{nv}(1+kt/(2v)+O(t^2))$. Thus $1/w-1/u\to k/(6\sqrt{n}{v}^{3/2})$. The approximation requires suitable tail regularity and is not a universal formula for distributions without a moment-generating function.

For a regular model with a suitable ancillary statistic $a$, the displayed expression approximates the conditional density of the maximum-likelihood estimator. The normalizing factor includes $(2\pi {)}^{-p/2}$ for a $p$-dimensional estimator, depending on convention. The observed information is evaluated at the sample's own fitted value. In a pure scale model the factor is proportional to $1/\hat{\theta }$; this Jacobian determinant can make the normalized approximation coincide with the exact conditional density.

For independent samples of size $n$ from exponential distributions with rates $\lambda$ and $\psi \lambda$, the maximum-likelihood estimators are $\hat{\lambda }=n/S$ and $\hat{\psi }=S/T$, where $S$ and $T$ are the respective sample sums. Their normalized P-star approximation is exact:Indeed, $S$ and $T$ have independent gamma distributions with shapes $n$ and respective rates $\lambda ,\psi \lambda$. The Jacobian determinant of their inverse transformation is $n^2/({\hat{\psi }}^2{\hat{\lambda }}^3)$, giving the displayed density. Alternatively the square root of the determinant of the fitted observed information is $n/(\hat{\psi }\hat{\lambda })$; multiplying by the exponential of the fitted log-likelihood difference gives the same shape. Its normalizing constant in the P-star approximation is $c_n=n^{2n-1}{e}^{-2n}/\Gamma (n{)}^2$. Integrating over $\hat{\lambda }$ givesso $\hat{\psi }/\psi$ has the F-distribution with degrees of freedom $(2n,2n)$.

For a size-$n$ sample from an exponential distribution of mean $\theta$, the likelihood in the maximum-likelihood estimator coordinate $v=\bar{Y}$ has difference $\ell (\theta ;v)-\ell (v;v)=n\log (v/\theta )-nv/\theta +n$, and the fitted observed information is $n/v^2$. Their P-star approximation is the displayed gamma density shape. Its normalization gives exactly the sampling density $n^n{v}^{n-1}{e}^{-nv/\theta }/(\Gamma (n){\theta }^n)$. This illustrates an exact transformation-model case rather than a general claim of exactness.

Fix an ancillary statistic $a$ and a parameter value at which the score function is evaluated. Express that score as a function $u(v,a)$ of the varying maximum-likelihood estimator coordinate $v$. The ordinary density change-of-variables formula applied to the P-star approximation gives the displayed density; several monotone branches require a sum. Its derivative is a sample-coordinate derivative, not generally the parameter observed information.

If a density has polynomially decaying tails on both half-lines, its moment-generating function is infinite at every nonzero real parameter: exponential growth defeats polynomial decay on the positive tail for a positive parameter, and on the negative tail for a negative parameter. Thus no open real tilting domain exists for a saddlepoint density approximation, even when the distribution has finite mean and variance and a central limit theorem applies.

The $n$th cumulant is the coefficient of $t^n/n!$ in the cumulant-generating function. The first two cumulants are the expected value and variance; the third is the third central moment.

The cumulant expansion is the Taylor expansion of the logarithm of a moment-generating function:If the moment-generating function is finite on a neighbourhood of zero, the logarithm is analytic on a sufficiently small neighbourhood and this is a convergent local expansion. The first terms areA formal series of moments is not automatically a convergent representation of a moment-generating function. Finitely many moments can instead justify finite-order expansions of the characteristic function and its logarithm near zero; exponential integrability is a separate hypothesis.

For a centered normal distribution of variance ${\sigma }^2$, $K_X(t)={\sigma }^2{t}^2/2$, so all cumulants beyond order two vanish. For a Poisson distribution of mean $\lambda$, $K_X(t)=\lambda (e^t-1)$, so every cumulant equals $\lambda$. independent sums add their cumulant-generating functions, making cumulants additive.

For a fluctuating interaction $V$, expanding a Boltzmann weight gives $\log \mathbb{E}{e}^{-\beta gV}=-\beta g\mathbb{E}V+{\beta }^2{g}^2\mathrm{Var}(V)/2+\cdots$ whenever the exponential moment exists near $g=0$. The logarithm thus organizes the expansion into cumulants rather than raw moments.

An Edgeworth expansion refines the central limit theorem by adding corrections determined by higher cumulants. Let $X_1,X_2,\dots$ be independent and identically distributed with mean zero, variance ${\sigma }^2>0$ and skewness ${\gamma }_1=\mathbb{E}{X}_1^3/{\sigma }^3$. For the standardized sum $Z_n=(X_1+\cdots +X_n)/(\sigma \sqrt{n})$, the first density correction isHere $\varphi$ is the standard normal density. This is an asymptotic statement for a sequence, not a universal density formula for one arbitrary random variable. Sufficient smoothness and moment conditions are needed; one strong sufficient setting is that the common probability density function is a non-degenerate Schwartz function. In that setting the displayed remainder is valid for each fixed $z$. Lattice-valued variables need additional corrections rather than this density formula.^[ref]

To identify the coefficient, expand the log characteristic function of the standardized sum:for fixed $t$. Exponentiation and Fourier inversion turn $(it{)}^3{e}^{-t^2/2}$ into the Hermite polynomial ${\mathrm{He}}_3(z)=z^3-3z$ times $\varphi (z)$. The smoothness assumptions justify inversion beyond this formal calculation. Higher-order terms use further cumulants and Hermite polynomials.

The approximation improves central asymptotics, but it need not be a nonnegative normalized density after truncation. In particular the cubic correction can become negative in a far tail, where a fixed-point expansion does not provide a small relative error. Inverting a distribution-function Edgeworth expansion produces the Cornish-Fisher expansion for fixed quantiles.

For a standardized iid sum with a regular Edgeworth expansion, the first distribution correction is proportional to $\varphi (t)(t^2-1)$ and is even. Subtracting the two endpoint distribution functions therefore cancels the $n^{-1/2}$ contribution. Fixed symmetric pivot cutoffs can consequently have $O(n^{-1})$ coverage error even though each one-sided approximation has error $O(n^{-1/2})$. This cancellation does not automatically hold for asymmetric or estimated random cutoffs.

The Cornish-Fisher expansion inverts an Edgeworth expansion to approximate quantiles. For a standardized iid sum, let ${\gamma }_1$ be the skewness of one summand and $z_{\alpha }$ the standard normal quantile. The first distribution function correction is $-{\gamma }_1(z^2-1)\varphi (z)/(6\sqrt{n})$. Substituting $y_{\alpha }=z_{\alpha }+c/\sqrt{n}$ and expanding $\Phi$ gives $c={\gamma }_1(z_{\alpha }^2-1)/6$. This is a fixed-quantile asymptotic, with $\alpha$ fixed in $(0,1)$.

The Legendre transform of a cumulant-generating function $K(\theta )=\log \mathbb{E}{e}^{\theta X}$ iswhere parameters with $K(\theta )=+\infty$ contribute nothing to the supremum. More generally this is the Legendre-Fenchel transform. It is convex because it is a supremum of affine functions of $x$, and is nonnegative because $K(0)=0$ allows the choice $\theta =0$.

If $K$ is differentiable and the supremum is attained at an interior parameter, its maximizer solves $K^{\prime }(\theta )=x$. For a Normal distribution with mean $\mu$ and variance ${\sigma }^2>0$, $K(\theta )=\mu \theta +{\sigma }^2{\theta }^2/2$, so $\theta =(x-\mu )/{\sigma }^2$ and $I(x)=(x-\mu {)}^2/(2{\sigma }^2)$. The zero cost occurs at the mean and the cost grows quadratically away from it.

For iid sample means, Cramér's theorem identifies this transform as the large-deviation Rate function when the Moment-generating function is finite on a neighbourhood of zero. The optimization also appears directly in a Chernoff bound: optimize the exponential Markov inequality over nonnegative parameters to bound an upper-tail probability. The transform converts exponential growth of a generating function into exponential decay of a rare-event probability.

For a uniform random sign $X\in \{-1,1\}$, $\psi (\lambda )=\log \cosh \lambda$ andfor $|x|\le 1$, with value $+\infty$ outside that interval.

For independent and identically distributed random variables with a moment-generating function finite on an open interval containing zero, the empirical mean satisfies a large deviation principle with Rate function given by the Legendre transform of a cumulant-generating function. The exponential-moment hypothesis is part of this theorem; mere integrability does not guarantee this full large-deviation principle.

For sample means of Poisson random variables of mean $\lambda >0$, the cumulant-generating function is $\lambda (e^{\theta }-1)$. Its Legendre-Fenchel transform is the displayed good rate function, with $0\log 0=0$, and is infinite for $x<0$. Near its zero, $I_P(\lambda +h)=h^2/(2\lambda )+O(h^3)$. This local quadratic explains the Poisson moderate deviation principle.

For independent and identically distributed exponential random variables of rate $\lambda >0$, the Sample mean has large-deviation speed $n$ and the displayed good rate function, infinite for $x\le 0$. The cumulant-generating function is $\log [\lambda /(\lambda -\theta )]$ for $\theta <\lambda$. Its Legendre-Fenchel transform has maximizing parameter $\theta =\lambda -1/x$. For $x\le 0$, sending $\theta \to -\infty$ gives infinite cost. The cost has its unique zero at $1/\lambda$ and diverges at both ends of its positive domain.

For independent unit-rate exponential random variables, the cumulant-generating function is $-\log (1-\theta )$ for $\theta <1$. Its Legendre transform of a cumulant-generating function is $I(x)=x-1-\log x$ for $x>0$ and infinite otherwise. The Cramér theorem gives the upper-tail logarithmic rate $-I(a)$ for $a>1$, and zero for $0\le a\le 1$. At the mean threshold the infima on both the open and closed upper half-lines are zero; below the mean the strong law of large numbers applies.

For independent identically distributed real variables with everywhere finite cumulant-generating function, let $\mu$ be the mean. If $a\ge \mu$ and $\mathbb{P}(X_1>a)>0$, then $\mathbb{P}(S_n/n\in [a,a+\epsilon )\mid S_n/n\ge a)\to 1$ for each $\epsilon >0$. The Legendre transform of a cumulant-generating function is finite at $a$, and a positive exponential parameter gives strictly greater cost at $a+\epsilon$. A Chernoff bound for the farther tail divided by the Cramér theorem lower estimate at $a$ decays exponentially. The support condition both ensures positive conditioning probabilities and avoids a vacuous conditioning event.

exponential tilting reweights a probability law in favour of larger or smaller values. Let $K(\theta )=\log {\mathbb{E}}_P{e}^{\theta X}$ be finite at the chosen parameter. Define the tilted law by its Radon-Nikodym derivativeIts total mass is one by construction. Positive $\theta$ favours larger $x$, while negative $\theta$ favours smaller $x$; it changes probabilities without creating values outside the original support.

For $\theta$ in the interior of the finite moment-generating-function domain, differentiation under the integral givesThe interior-domain condition supplies the integrable exponential bounds needed for these derivatives. Endpoint tilts can exist without a finite tilted mean, so existence of a normalized tilt alone is insufficient for the mean formula.

For a Bernoulli distribution with success probability $p\in (0,1)$, the tilted success probability is $p_{\theta }=p{e}^{\theta }/(1-p+p{e}^{\theta })$: tilting adds $\theta$ to its log odds. A Normal distribution $N(\mu ,{\sigma }^2)$ tilts to $N(\mu +{\sigma }^2\theta ,{\sigma }^2)$ by completing the square. These calculations explain why exponential tilting is useful for making a rare Sample mean typical while retaining an explicit Likelihood ratio.

Suppose $\varphi (\lambda )=\mathbb{E}{e}^{\lambda X}<\infty$ for all $\lambda \ge 0$. Under the exponential tilting $d{\mu }^{\lambda }=e^{\lambda x}d\mu /\varphi (\lambda )$, $X$ is an integrable random variable whenever $\lambda >0$: its negative exponential tail is bounded, and its positive contribution is dominated by a larger positive exponential moment. Differentiation under the integral sign gives the displayed mean. At the endpoint $\lambda =0$, the positive part of $X$ is integrable, but the negative part need not be. The right derivative ${\varphi }_{+}^{\prime }(0)=\mathbb{E}X$ is valid as an extended value, possibly $-\infty$; a finite mean at zero requires an extra negative-tail integrability assumption.