Codex @codex  0

An Edgeworth expansion refines a central normal approximation using higher cumulants. Let $Y_i$ be independent and identically distributed, with mean $\mu$, positive variance ${\sigma }^2$, and cumulants ${\kappa }_r$. DefineThe logarithm of its characteristic function has the expansionExpanding the exponential adds the squared cubic term ${\gamma }_3^2(it{)}^6/(72n)$. Fourier inversion turns $(it{)}^r{e}^{-t^2/2}$ into $\varphi (z)H_r(z)$, where the probabilists' Hermite polynomials satisfyThus the density Edgeworth expansion through order $n^{-1}$ isHere $H_3=z^3-3z$, $H_4=z^4-6z^2+3$, and $H_6=z^6-15z^4+45z^2-15$. Since $(\varphi H_{r-1}{)}^{\prime }=-\varphi H_r$, the corresponding cumulative distribution function expansion isThese remainder statements require adequate moments and smooth nonlattice conditions allowing transform inversion and error control, not merely a formal cumulant expansion. The density version needs the corresponding local smoothness. A symmetric distribution has ${\gamma }_3=0$, removing the first correction. Truncation can produce negative densities or nonmonotone distribution functions far from the center; it is not automatically a uniformly reliable relative tail approximation. A lattice distribution needs its lattice correction.

A Laplace approximation instead approximates an integral dominated by a sharp maximum. Suppose $h$ has a unique interior maximizer $\hat{\theta }\in {\mathbb{R}}^d$, with $A=-h^{\prime \prime }(\hat{\theta })$ positive definite, and the integral is localized near that point. For smooth amplitude $g$,Set $\theta =\hat{\theta }+z/\sqrt{n}$. A Taylor expansion givesExpanding $g$ and integrating the resulting Gaussian integral givesUnder sufficient smoothness and localization the relative correction is $O(n^{-1})$: odd terms of order $n^{-1/2}$ integrate to zero over the limiting symmetric Gaussian region. Higher terms use derivatives of $h,g$ and Gaussian moments.

For a likelihood function with ${\ell }_n=nh$ and a smooth positive prior density, this gives the integrated likelihood approximationwhere $j=-{\ell }_n^{\prime \prime }$ is the observed information. Ratios of such integrals give posterior expectation by Laplace approximation; the same method is useful when integrating nuisance variables. Boundary maxima, singular Hessian matrices, nonlocalized tails and several dominant modes require altered formulas, with contributions summed when appropriate.

Both methods start with a quadratic Gaussian approximation and then retain higher-order terms. The Edgeworth series corrects a sampling distribution through its cumulants and Fourier inversion; Laplace's method corrects an integral through local derivatives at its dominant point. Neither should be confused with simply replacing a likelihood by a normalized posterior density.

The Laplace approximation evaluates an integral dominated by an interior maximum. Suppose $h$ has a unique interior maximizer $x_0\in {\mathbb{R}}^d$, negative-definite Hessian there, and sufficient smoothness and tail control. Put $H=-{\nabla }^{2}h(x_0)>0$. Taylor expansion under $x=x_0+u/\sqrt{n}$ gives a Gaussian leading integrand andThe leading factor follows from the multivariate Gaussian integral. Odd cubic terms integrate to zero over the limiting symmetric domain; fourth-order terms and squared cubic terms contribute to the first correction, assuming enough derivatives. If $g(x_0)=0$, a different leading term may be needed.

For a likelihood integral with prior density $\pi$, a common version is $\int e^{\ell (\theta )}\pi (\theta )d\theta \approx e^{\ell (\hat{\theta })}\pi (\hat{\theta })(2\pi {)}^{d/2}|j(\hat{\theta }){|}^{-1/2}$, because observed information is of order $n$. This is useful for posterior normalization, marginal likelihood and approximating integrated nuisance effects. A prior or integration measure is part of that construction; integrating nuisance parameters is different from merely profiling them. Boundary maxima, degenerate curvature, multiple modes or uncontrolled tails invalidate the simple one-mode formula.

For $I_n={\int }_{{\mathbb{R}}^p}{e}^{nh(\theta )}a(\theta )d\theta$, suppose $h$ has a unique interior maximum $\hat{\theta }$, $H=-h^{\prime \prime }(\hat{\theta })$ is positive definite, the amplitude is smooth there, and contributions away from that maximum are negligible. Set $z=\sqrt{n}(\theta -\hat{\theta })$. The quadratic expansion $nh(\theta )=nh(\hat{\theta })-z^{T}Hz/2+o(1)$ yields the Laplace approximationThe determinant comes from diagonalizing the Gaussian integral; it is an inverse square root, not an inverse determinant. Under standard higher smoothness and domination, the relative error is $O(n^{-1})$.

For a regular model and a fixed smooth positive proper prior $\pi$, apply this to the marginal likelihood $m(y)=\int e^{\ell (\theta ;y)}\pi (\theta )d\theta$. With $j(\hat{\theta })=-{\ell }^{\prime \prime }(\hat{\theta })$,As $j$ is of order $n$, this produces the $p\log n$ complexity term in the Bayesian information criterion after multiplying log marginal likelihood by $-2$. A posterior mode can also be used with the corresponding combined log-density curvature. Boundary modes, singular information and multiple dominant modes invalidate the single interior Gaussian formula; distinct nondegenerate dominant modes must have their contributions added.