Cumulant-generating function (${\psi }_X(\theta )$)

The cumulant-generating function is ${\psi }_X(\theta )=\log \mathbb{E}{e}^{\theta X}$, interpreted as an extended-real convex function where the exponential moment may be infinite.

For independent identically distributed continuous random variables, let $K(t)=\log \mathbb{E}{e}^{tY}$ be finite near zero. At a mean value $x$ reached by $K^{\prime }(\hat{t})=x$ in the interior of that domain, the displayed leading approximation describes the density of the sum $S_n$. Exponential tilting by $\hat{t}$ puts the sum at its tilted mean; a local normal approximation gives the denominator, and undoing the tilt gives the exponential factor. Under suitable smooth nonlattice regularity, at fixed interior $x$ the exact density is the displayed quantity times $1+O(n^{-1})$. This statistical approximation is an application of the saddle-point approximation to a transform inversion integral. It requires more than the existence of a few ordinary moments.

For independent unit-scale Laplace distributions, the cumulant-generating function is $K(t)=-\log (1-t^2)$ for $|t|<1$. The saddlepoint density approximation solves $K^{\prime }(t)=y$, giving $t=y/(1+\sqrt{1+y^2})$. At this tilt, $K^{\prime \prime }(t)=D(1+D)$ and $ty=D-1$, giving the displayed probability density function approximation for the sample mean. Its value at zero is $\sqrt{n/(4\pi )}$; it need not integrate to one before normalization.

For a sample mean from an independent identically distributed sample, let $K(t)$ be the cumulant-generating function of one observation and solve $K^{\prime }(t)=x$. Put $w=\mathrm{sgn}(t)\sqrt{2n(tx-K(t))}$ and $u=t\sqrt{n{K}^{\prime \prime }(t)}$. Under the smooth, nonlattice conditions for a saddlepoint density approximation, the corresponding approximation to the cumulative distribution function iswhere $\Phi$ and $\varphi$ are the standard normal distribution functions. The apparent singularity at $t=0$ is removable. Write $v=K^{\prime \prime }(0)$ and $k=K^{\prime \prime \prime }(0)$; Taylor expansion gives $w=t\sqrt{nv}(1+kt/(3v)+O(t^2))$ and $u=t\sqrt{nv}(1+kt/(2v)+O(t^2))$. Thus $1/w-1/u\to k/(6\sqrt{n}{v}^{3/2})$. The approximation requires suitable tail regularity and is not a universal formula for distributions without a moment-generating function.

For a regular model with a suitable ancillary statistic $a$, the displayed expression approximates the conditional density of the maximum-likelihood estimator. The normalizing factor includes $(2\pi {)}^{-p/2}$ for a $p$-dimensional estimator, depending on convention. The observed information is evaluated at the sample's own fitted value. In a pure scale model the factor is proportional to $1/\hat{\theta }$; this Jacobian determinant can make the normalized approximation coincide with the exact conditional density.

For independent samples of size $n$ from exponential distributions with rates $\lambda$ and $\psi \lambda$, the maximum-likelihood estimators are $\hat{\lambda }=n/S$ and $\hat{\psi }=S/T$, where $S$ and $T$ are the respective sample sums. Their normalized P-star approximation is exact:Indeed, $S$ and $T$ have independent gamma distributions with shapes $n$ and respective rates $\lambda ,\psi \lambda$. The Jacobian determinant of their inverse transformation is $n^2/({\hat{\psi }}^2{\hat{\lambda }}^3)$, giving the displayed density. Alternatively the square root of the determinant of the fitted observed information is $n/(\hat{\psi }\hat{\lambda })$; multiplying by the exponential of the fitted log-likelihood difference gives the same shape. Its normalizing constant in the P-star approximation is $c_n=n^{2n-1}{e}^{-2n}/\Gamma (n{)}^2$. Integrating over $\hat{\lambda }$ givesso $\hat{\psi }/\psi$ has the F-distribution with degrees of freedom $(2n,2n)$.

For a size-$n$ sample from an exponential distribution of mean $\theta$, the likelihood in the maximum-likelihood estimator coordinate $v=\bar{Y}$ has difference $\ell (\theta ;v)-\ell (v;v)=n\log (v/\theta )-nv/\theta +n$, and the fitted observed information is $n/v^2$. Their P-star approximation is the displayed gamma density shape. Its normalization gives exactly the sampling density $n^n{v}^{n-1}{e}^{-nv/\theta }/(\Gamma (n){\theta }^n)$. This illustrates an exact transformation-model case rather than a general claim of exactness.

Fix an ancillary statistic $a$ and a parameter value at which the score function is evaluated. Express that score as a function $u(v,a)$ of the varying maximum-likelihood estimator coordinate $v$. The ordinary density change-of-variables formula applied to the P-star approximation gives the displayed density; several monotone branches require a sum. Its derivative is a sample-coordinate derivative, not generally the parameter observed information.

If a density has polynomially decaying tails on both half-lines, its moment-generating function is infinite at every nonzero real parameter: exponential growth defeats polynomial decay on the positive tail for a positive parameter, and on the negative tail for a negative parameter. Thus no open real tilting domain exists for a saddlepoint density approximation, even when the distribution has finite mean and variance and a central limit theorem applies.

The $n$th cumulant is the coefficient of $t^n/n!$ in the cumulant-generating function. The first two cumulants are the expected value and variance; the third is the third central moment.

The cumulant expansion writeswhere ${\kappa }_n$ is the $n$th cumulant. Applied to a fluctuating interaction, it expands the logarithm of the averaged Boltzmann weight in connected expectation values.

An Edgeworth expansion approximates a probability density function by a normal distribution multiplied by Hermite polynomials encoding higher cumulants. For a centered variable of variance ${\sigma }^2$ and third cumulant ${\kappa }_3$,The correction follows from inverse-transforming the cubic term of the cumulant-generating function. It is an asymptotic approximation near the bulk of the distribution; truncation can give a negative result in far tails.

For a standardized iid sum with a regular Edgeworth expansion, the first distribution correction is proportional to $\varphi (t)(t^2-1)$ and is even. Subtracting the two endpoint distribution functions therefore cancels the $n^{-1/2}$ contribution. Fixed symmetric pivot cutoffs can consequently have $O(n^{-1})$ coverage error even though each one-sided approximation has error $O(n^{-1/2})$. This cancellation does not automatically hold for asymmetric or estimated random cutoffs.

The Cornish-Fisher expansion inverts an Edgeworth expansion to approximate quantiles. For a standardized iid sum, let ${\gamma }_1$ be the skewness of one summand and $z_{\alpha }$ the standard normal quantile. The first distribution function correction is $-{\gamma }_1(z^2-1)\varphi (z)/(6\sqrt{n})$. Substituting $y_{\alpha }=z_{\alpha }+c/\sqrt{n}$ and expanding $\Phi$ gives $c={\gamma }_1(z_{\alpha }^2-1)/6$. This is a fixed-quantile asymptotic, with $\alpha$ fixed in $(0,1)$.

The Legendre transform is the convex function

For a uniform random sign $X\in \{-1,1\}$, $\psi (\lambda )=\log \cosh \lambda$ andfor $|x|\le 1$, with value $+\infty$ outside that interval.

For independent and identically distributed random variables with a moment-generating function finite on an open interval containing zero, the empirical mean satisfies a large deviation principle with rate function given by the Legendre transform of a cumulant-generating function. The exponential-moment hypothesis is part of this theorem; mere integrability does not guarantee this full large-deviation principle.

For sample means of Poisson random variables of mean $\lambda >0$, the cumulant-generating function is $\lambda (e^{\theta }-1)$. Its Legendre-Fenchel transform is the displayed good rate function, with $0\log 0=0$, and is infinite for $x<0$. Near its zero, $I_P(\lambda +h)=h^2/(2\lambda )+O(h^3)$. This local quadratic explains the Poisson moderate deviation principle.

For independent and identically distributed exponential random variables of rate $\lambda >0$, the sample mean has large-deviation speed $n$ and the displayed good rate function, infinite for $x\le 0$. The cumulant-generating function is $\log [\lambda /(\lambda -\theta )]$ for $\theta <\lambda$. Its Legendre-Fenchel transform has maximizing parameter $\theta =\lambda -1/x$. For $x\le 0$, sending $\theta \to -\infty$ gives infinite cost. The cost has its unique zero at $1/\lambda$ and diverges at both ends of its positive domain.

For independent unit-rate exponential random variables, the cumulant-generating function is $-\log (1-\theta )$ for $\theta <1$. Its Legendre transform of a cumulant-generating function is $I(x)=x-1-\log x$ for $x>0$ and infinite otherwise. The Cramér theorem gives the upper-tail logarithmic rate $-I(a)$ for $a>1$, and zero for $0\le a\le 1$. At the mean threshold the infima on both the open and closed upper half-lines are zero; below the mean the strong law of large numbers applies.

For independent identically distributed real variables with everywhere finite cumulant-generating function, let $\mu$ be the mean. If $a\ge \mu$ and $\mathbb{P}(X_1>a)>0$, then $\mathbb{P}(S_n/n\in [a,a+\epsilon )\mid S_n/n\ge a)\to 1$ for each $\epsilon >0$. The Legendre transform of a cumulant-generating function is finite at $a$, and a positive exponential parameter gives strictly greater cost at $a+\epsilon$. A Chernoff bound for the farther tail divided by the Cramér theorem lower estimate at $a$ decays exponentially. The support condition both ensures positive conditioning probabilities and avoids a vacuous conditioning event.

Exponential tilting by $\theta$ replaces a law $P$ byUnder the tilted law, the mean is ${\psi }^{\prime }(\theta )$ whenever the derivative exists.

Suppose $\varphi (\lambda )=\mathbb{E}{e}^{\lambda X}<\infty$ for all $\lambda \ge 0$. Under the exponential tilting $d{\mu }^{\lambda }=e^{\lambda x}d\mu /\varphi (\lambda )$, $X$ is an integrable random variable whenever $\lambda >0$: its negative exponential tail is bounded, and its positive contribution is dominated by a larger positive exponential moment. Differentiation under the integral sign gives the displayed mean. At the endpoint $\lambda =0$, the positive part of $X$ is integrable, but the negative part need not be. The right derivative ${\varphi }_{+}^{\prime }(0)=\mathbb{E}X$ is valid as an extended value, possibly $-\infty$; a finite mean at zero requires an extra negative-tail integrability assumption.