Past exam of the mathematics course of the University of Cambridge / 2005 / iii / Paper 42 / 6

An Edgeworth expansion refines the central limit theorem by incorporating higher cumulants. For iid centred observations with variance ${\sigma }^2$, standardized sum $Z_n={\sum }_i{X}_i/(\sigma \sqrt{n})$, standardized third cumulant ${\gamma }_1={\kappa }_3/{\sigma }^3$ and fourth cumulant ${\gamma }_2={\kappa }_4/{\sigma }^4$, its central-region density expansion isunder sufficient moment and smoothness/nonlattice conditions. Here $H_3=z^3-3z$, $H_4=z^4-6z^2+3$, $H_6=z^6-15z^4+45z^2-15$ are probabilists' Hermite polynomials. Expanding the characteristic exponent gives the cubic and quartic terms, while squaring the cubic term in the exponential gives the $H_6$ correction. Integrating the first correction, using $(\varphi H_{k-1}{)}^{\prime }=-\varphi H_k$, givesThus skewness determines the first departure from the normal approximation; inversion gives a Cornish-Fisher expansion for quantiles. These are asymptotic expansions, not guaranteed nonnegative densities in extreme tails. Lattice data require continuity/lattice corrections, and moment assumptions alone do not justify a uniform smooth-density expansion.

Parameters $\psi$ and $\lambda$ are orthogonal parameters if the cross block of the expected Fisher information matrix is zero throughout the relevant parameter region:This is a statement about expected information, not a requirement that every observed cross derivative vanish. For regular maximum-likelihood estimators, $\sqrt{n}(\hat{\theta }-\theta )$ has limiting covariance equal to the inverse per-observation Fisher information matrix. Block diagonality therefore gives asymptotically independent fitted interest and nuisance parameters. The efficient information $I_{\psi \psi }-I_{\psi \lambda }{I}_{\lambda \lambda }^{-1}{I}_{\lambda \psi }$ reduces to $I_{\psi \psi }$ at an orthogonal parametrization, although nuisance estimation can still matter at higher orders.

For scalar interest and nuisance, retain $\psi$ and write the old nuisance as $\lambda =\lambda (\psi ,\eta )$. The transformed cross information is ${\lambda }_{\eta }(I_{\psi \lambda }+{\lambda }_{\psi }{I}_{\lambda \lambda })$. Hence an orthogonal nuisance coordinate can locally be found fromsubject to local solvability and a nonsingular coordinate change. Global coordinates need not follow from this local construction. For the normal distribution with separately varying mean and variance parameters $(\mu ,v)$, the scores are $(Y-\mu )/v$ and $-1/(2v)+(Y-\mu {)}^2/(2v^2)$; their product has zero expectation by symmetry. Thus the per-observation information is diagonal with entries $1/v$ and $1/(2v^2)$.

For $I_n={\int }_{{\mathbb{R}}^p}{e}^{nh(\theta )}a(\theta )d\theta$, suppose $h$ has a unique interior maximum $\hat{\theta }$, $H=-h^{\prime \prime }(\hat{\theta })$ is positive definite, the amplitude is smooth there, and contributions away from that maximum are negligible. Set $z=\sqrt{n}(\theta -\hat{\theta })$. The quadratic expansion $nh(\theta )=nh(\hat{\theta })-z^{T}Hz/2+o(1)$ yields the Laplace approximationThe determinant comes from diagonalizing the Gaussian integral; it is an inverse square root, not an inverse determinant. Under standard higher smoothness and domination, the relative error is $O(n^{-1})$.

For a regular model and a fixed smooth positive proper prior $\pi$, apply this to the marginal likelihood $m(y)=\int e^{\ell (\theta ;y)}\pi (\theta )d\theta$. With $j(\hat{\theta })=-{\ell }^{\prime \prime }(\hat{\theta })$,As $j$ is of order $n$, this produces the $p\log n$ complexity term in the Bayesian information criterion after multiplying log marginal likelihood by $-2$. A posterior mode can also be used with the corresponding combined log-density curvature. Boundary modes, singular information and multiple dominant modes invalidate the single interior Gaussian formula; distinct nondegenerate dominant modes must have their contributions added.

Let $W=2[\ell (\hat{\theta })-\ell (\tilde{\theta })]$ be a regular likelihood-ratio test statistic for $r$ smooth restrictions, so its leading null law is ${\chi }_r^2$. Suppose a higher-order calculation gives ${\mathbb{E}}_{0}W=r+a/n+O(n^{-2})$. The Bartlett correction rescales the statistic:Its mean is $r+O(n^{-2})$, eliminating the first-order mean error. In regular likelihood models satisfying the usual Bartlett expansion, the same scaling removes the leading $n^{-1}$ distributional error against the chi-square reference law, leaving an $O(n^{-2})$ error for fixed tail probabilities. Matching a mean alone would not establish this distributional conclusion for an arbitrary statistic.

The coefficient depends on the model, null restrictions and nuisance parameters; it can be obtained from higher derivatives and cumulants of the log-likelihood, or estimated by null simulation when an analytic expression is inconvenient. Nuisance parameters require their appropriate null values or estimates. A correction improves finite-sample calibration within regular asymptotic theory; it does not turn a boundary-parameter or unidentifiable problem into a regular chi-square problem.

In statistical decision theory, the statistical invariance principle says that a procedure should respect transformations that leave the inferential problem unchanged. If a group action preserves the model and carries each of the null and alternative parameter sets into itself, an invariant test has rejection probability function $\phi (gy)=\phi (y)$. A maximal invariant identifies its orbits, so every invariant test factors through that statistic: define its value at an orbit using any representative; invariance makes the definition independent of the representative. If the group is transitive on the null parameter set, the null law of any invariant statistic is free of that nuisance parameter.

For example, testing a normal mean $\mu ={\mu }_0$ with unknown scale, positive transformations $y_i-{\mu }_0\mapsto b(y_i-{\mu }_0)$ preserve a one-sided mean alternative. The statistic $\sqrt{n}(\bar{Y}-{\mu }_0)/s$, with the usual unbiased sample-variance scale $s$, is invariant and has the Student t-distribution with $n-1$ degrees of freedom under the null. For estimation the corresponding requirement is an equivariant estimator, together with a loss function invariant under simultaneous transformation of decision and parameter. Invariance alone does not prove unrestricted optimality or supply a unique best procedure.

There is a distinct probabilistic use of the term: the Donsker invariance principle states that centred iid partial sums, normalized by $\sigma \sqrt{n}$, converge as processes on $[0,1]$ to Brownian motion. Its empirical-process version gives a Brownian bridge, used by goodness-of-fit tests in the next note. These functional limit theorems should be distinguished from the group-based inferential principle.

Population influence functions describe infinitesimal contamination; finite-sample analogues examine actual changes to an empirical sample. For its empirical distribution $F_n$, add an observation $z$ and define the sensitivity curveUnder suitable differentiability it approximates the influence function. A replacement version is ${\mathrm{RC}}_{n,i}(z)=n[T_n(y_1,\dots ,z,\dots ,y_n)-T_n(y)]$, approximately $\mathrm{IF}(z;T,F_n)-\mathrm{IF}(y_i;T,F_n)$. Their supremum magnitudes quantify finite one-observation sensitivity.

The empirical influence function instead substitutes $F_n$ directly in $\mathrm{IF}(z;T,F)$. For a differentiable M-estimator this gives $-\psi (z,T(F_n))/[n^{-1}{\sum }_i{\partial }_{\theta }\psi (y_i,T(F_n))]$. For quantile functionals, a discrete unsmoothed empirical law may not admit the required density derivative; a sensitivity curve or a smoothed empirical law is then preferable.

Global contamination is measured by finite-sample maximum bias $b_n(m;y)={\sup }_{z:d_H(z,y)\le m}|T_n(z)-T_n(y)|$, where $d_H$ counts replacements. Under the convention of the first fraction that causes unbounded bias,This is the finite-sample replacement breakdown point. The largest replacement fraction still guaranteed bounded is one grid step smaller, so the conventions must be specified. One replacement sends a sample mean arbitrarily far, giving $1/n$, while the ordinary scalar sample median has first-breaking fraction $\lceil n/2\rceil /n$ with the usual midpoint convention for even samples. Its limiting value is $1/2$. Local influence and global breakdown answer different questions; a bounded influence function is not by itself a proof of a high breakdown point.

For a fully specified continuous null distribution $F_0$, compare the empirical distribution function $F_n$ with $F_0$. The probability integral transform makes $F_0(Y_i)$ iid uniform under the null, giving distribution-free calibration. Three common discrepancies areThese define the Kolmogorov-Smirnov test, Cramér–von Mises criterion and Anderson–Darling test, respectively. Reject for large values. The supremum detects the largest discrepancy, the unweighted integral combines discrepancies throughout the distribution, and the reciprocal weight emphasizes the tails.

The Donsker theorem for empirical distribution functions gives $\sqrt{n}(F_n-F_0)\Rightarrow B^0\circ F_0$ for a Brownian bridge $B^0$, with covariance $\min (u,v)-uv$. Consequently the standard continuous-null limits are ${\sup }_{0\le u\le 1}|B^0(u)|$, ${\int }_0^1{B}^0(u{)}^{2}du$, and ${\int }_0^1{B}^0(u{)}^2/[u(1-u)]du$ for $\sqrt{n}{D}_n,C_n,A_n$, respectively; the weighted last limit also requires control near the endpoints. Exact finite-sample or asymptotic null quantiles provide critical values.

If parameters in $F_{\theta }$ are estimated from the same observations, these fully specified-null critical values generally fail. A first-order expansion isso the estimated parameter changes the limiting process. One must use the fitted-model null law, adjusted asymptotic theory, or a parametric bootstrap that re-estimates the parameters in each replicate.

For a regular iid model with a fixed-dimensional interior parameter ${\theta }_0$, identifiability, sufficient differentiability and moment domination, and nonsingular per-observation Fisher information $I({\theta }_0)$, a consistent maximum-likelihood estimator solves the score equation. Expanding it giveswhere $s_{\theta }={\partial }_{\theta }\log f_{\theta }$. The score has mean zero and covariance $I$ under the regularity assumptions. This proves the usual large-sample information bound and asymptotic efficiency within regular parametric inference. The observed curvature $j(\hat{\theta })/n$ and the empirical score covariance consistently estimate $I$ when the model is correctly specified.

For $r$ smooth constraints $h(\theta )=0$ with full-row-rank derivative $H$, the Wald test usesThe likelihood-ratio test and score test, evaluated using the constrained fit and nuisance-adjusted efficient information, have the same first-order chi-square law. A quadratic local likelihood expansion explains their asymptotic equivalence. Profile likelihood supplies inference about a parameter of interest with fitted nuisance parameters, and the corresponding chi-square cutoffs give approximate confidence regions.

These conclusions depend on the stated regularity: parameters at a boundary, nonidentifiability, singular information, changing dimension, or parameter-dependent support can give different rates and limits. Under misspecification, the fitted parameter generally approaches a pseudo-true value and its variance is a sandwich covariance matrix, not simply inverse Fisher information.