Past exam of the mathematics course of the University of Cambridge / 2003 / iii / Paper 41 / 6

A Bartlett correction improves calibration of a regular likelihood-ratio test statistic $W=2\{\ell (\hat{\theta })-\ell ({\hat{\theta }}_0)\}$, whose null limit is ${\chi }_q^2$ for $q$ tested restrictions. Ifthe corrected statistic is $\boxed{W_B=W/(1+b/n)}$. Its null mean is $q+O(n^{-2})$ and it is compared with the same limiting chi-squared distribution. The coefficient is model-specific, may involve nuisance parameters, and can be calculated from likelihood cumulants or a suitable null simulation; substitution must respect the asymptotic regime.

Under the regular higher-order likelihood expansions and identities that establish Bartlett correctability, this rescaling removes the leading $O(n^{-1})$ distributional error and can give $O(n^{-2})$ chi-square calibration error. Matching one moment by itself does not prove that assertion for an arbitrary statistic, and nonregular boundary or singular models need different treatment.

For illustration, testing a normal variance with unknown mean gives $W=n[-\log (U/n)+U/n-1]$ with $U\sim {\chi }_{n-1}^2$. The gamma logarithmic moment yieldsThus the Bartlett correction for a normal variance with unknown mean uses $b=11/6$, not the correction for the different known-mean model. This example also distinguishes the relative coefficient $b$ from the full divisor $1+b/n$.

For parameter of interest $\psi$ and nuisance parameter $\lambda$, the profile likelihood has log form ${\ell }_p(\psi )=\ell (\psi ,{\hat{\lambda }}_{\psi })$, where ${\hat{\lambda }}_{\psi }$ is the constrained nuisance maximum. Profiling optimizes over the nuisance dimension but does not account for its local volume or conditioning geometry. The modified profile likelihood corrects this loss.

Use ancillary sample coordinates $(\hat{\psi },\hat{\lambda },a)$, and let $j_{\lambda \lambda }(\psi ,{\hat{\lambda }}_{\psi })=-{\ell }_{\lambda \lambda }$ be the nuisance block of observed information. Holding $\hat{\psi },a$ fixed, put $D(\psi )=|\partial {\hat{\lambda }}_{\psi }/\partial \hat{\lambda }|$. The Barndorff-Nielsen modified profile likelihood is, up to data-only factors,This combines a nuisance-information determinant and a data-coordinate Jacobian determinant. Implicit differentiation of ${\ell }_{\lambda }(\psi ,{\hat{\lambda }}_{\psi })=0$ gives $\partial {\hat{\lambda }}_{\psi }/\partial \hat{\lambda }=j_{\lambda \lambda }^{-1}{\ell }_{\lambda ;\hat{\lambda }}$, so equivalentlyThe semicolon denotes differentiation in the varying sample coordinate, rather than a parameter derivative. This form also explains the positive adjustment in Question 2: in that canonical conditional problem the mixed sample derivative is data-only, leaving the positive half-log curvature. The adjustment sign cannot be copied between unrelated constructions without tracking their Jacobians.

In an appropriate orthogonal nuisance formulation, the Cox-Reid adjusted profile likelihood uses ${\ell }_p-\frac{1}{2}\log |j_{\lambda \lambda }|$. It is a useful simpler adjustment, but discarding $D$ is not universally legitimate. For normal data with interest ${\sigma }^2$ and nuisance $\mu$, the constrained and full nuisance fits both equal $\bar{X}$, so $D=1$ and $j_{\mu \mu }=n/{\sigma }^2$. Thusrecovering the residual degrees of freedom and fitted variance ${\sum }_i(X_i-\bar{X}{)}^2/(n-1)$. These adjustments target nuisance-related bias and higher-order likelihood inference; they are approximations to conditional or marginal constructions, not an assertion that profiling has become an exact full sampling density.

The Laplace approximation evaluates an integral dominated by an interior maximum. Suppose $h$ has a unique interior maximizer $x_0\in {\mathbb{R}}^d$, negative-definite Hessian there, and sufficient smoothness and tail control. Put $H=-{\nabla }^{2}h(x_0)>0$. Taylor expansion under $x=x_0+u/\sqrt{n}$ gives a Gaussian leading integrand andThe leading factor follows from the multivariate Gaussian integral. Odd cubic terms integrate to zero over the limiting symmetric domain; fourth-order terms and squared cubic terms contribute to the first correction, assuming enough derivatives. If $g(x_0)=0$, a different leading term may be needed.

For a likelihood integral with prior density $\pi$, a common version is $\int e^{\ell (\theta )}\pi (\theta )d\theta \approx e^{\ell (\hat{\theta })}\pi (\hat{\theta })(2\pi {)}^{d/2}|j(\hat{\theta }){|}^{-1/2}$, because observed information is of order $n$. This is useful for posterior normalization, marginal likelihood and approximating integrated nuisance effects. A prior or integration measure is part of that construction; integrating nuisance parameters is different from merely profiling them. Boundary maxima, degenerate curvature, multiple modes or uncontrolled tails invalidate the simple one-mode formula.

In a transformation model, a group acts on data $y$ and parameters compatibly: $Y\sim P_{\theta }$ implies $gY\sim P_{g\theta }$. A statistic $A$ is invariant if $A(gy)=A(y)$, and is a maximal invariant if equality of its values also implies the samples lie on the same group orbit. A maximal invariant records all information unchanged by the transformations; measurable invariant procedures therefore factor through it, subject to the ordinary orbit-space measurability conditions.

An equivariant estimator satisfies $T(gy)=gT(y)$ for the induced action on the quantity estimated. For the positive affine location-scale group, $gy=a+by$ with $b>0$, this means $T_{\mathrm{loc}}(a+by)=a+b{T}_{\mathrm{loc}}(y)$ and $T_{\mathrm{scale}}(a+by)=b{T}_{\mathrm{scale}}(y)$.

For a nonconstant ordered sample, take $\bar{y}$ and $s^2=n^{-1}{\sum }_i(y_i-\bar{y}{)}^2>0$. Thenis a maximal invariant: it is unchanged by the positive affine group, and if $A(y)=A(z)$ then $z_i=\bar{z}+(s_z/s_y)(y_i-\bar{y})$, explicitly recovering the transformation. Both mean and standard deviation are equivariant. If the whole model varies only by location and positive scale, the normalized sample's distribution is independent of those two parameters, giving an ancillary statistic. It may still retain information about additional shape parameters.

Equivariance yields a coherent transformation rule and, under an invariant loss, can simplify risk comparison to a reference parameter. It does not on its own establish optimality. Normalization by a parameter-space estimator with a nontrivial stabilizer requires quotienting that remaining action; otherwise an apparently standardized statistic need not be maximal.

Partition a regular parameter into interest $\psi$ and nuisance $\lambda$. Parameter orthogonality means the cross-block of the expected Fisher information matrix vanishes:It is expected-information orthogonality, not a demand that every observed cross derivative be zero. To first order, the asymptotic normal distribution of the joint maximum-likelihood estimator has independent interest and nuisance blocks. Its efficient interest information simplifies from the Schur complement $I_{\psi \psi }-I_{\psi \lambda }{I}_{\lambda \lambda }^{-1}{I}_{\lambda \psi }$ to $I_{\psi \psi }$.

For scalar interest, new nuisance coordinates $\eta$ can locally be chosen by the local orthogonal nuisance reparametrizationIndeed the new interest score is $U_{\psi }+({\partial }_{\psi }\lambda {)}^{\top }{U}_{\lambda }$, and its expected product with the transformed nuisance score is zero by this equation. Smoothness, nonsingular nuisance information and suitable initial coordinates provide the local ordinary-differential-equation construction. Simultaneous orthogonalization for several interest coordinates may face integrability constraints.

For normal location and variance $(\mu ,{\sigma }^2)$, the cross information is zero: the location score is proportional to the centered observation and the variance score to its squared value minus its expectation, whose product has zero expectation by normal symmetry. This is a useful orthogonal parameterization. In general orthogonality reduces nuisance sensitivity and motivates adjusted profile constructions, but it implies neither exact estimator independence nor an exact nuisance-free conditional likelihood.

A one-sample U-statistic of fixed degree $m$ has a symmetric U-statistic kernel $h$ and averages it over all distinct subsets of an iid sample:It is unbiased for $\theta =\mathbb{E}h(X_1,\dots ,X_m)$. With finite second moment, define the first Hoeffding projection $h_1(x)=\mathbb{E}[h(x,X_2,\dots ,X_m)]-\theta$. Orthogonality of the higher projections givesIf ${\zeta }_1=\mathrm{Var}(h_1(X))>0$, the central limit theorem therefore proves the U-statistic central limit theoremEquivalently the influence function of a U-statistic is $m{h}_1(x)$, obtained by differentiating its population functional $\int hd{F}^m$ in each distribution argument. If ${\zeta }_1=0$, the root-$n$ limit degenerates and the second or higher projection determines a different, often nonnormal, limit.

For example, $h(x,y)=(x-y{)}^2/2$ gives the unbiased sample variance with denominator $n-1$. Its first projection is $[(x-\mu {)}^2-{\sigma }^2]/2$, so the influence function is $(x-\mu {)}^2-{\sigma }^2$. When the fourth moment is finite, the leading variance of the sample variance is $(\mathbb{E}(X-\mu {)}^4-{\sigma }^4)/n$. This example connects unbiased symmetric estimation to the functional influence calculation.

Several finite-sample counterparts of the influence function should be distinguished. The empirical influence function substitutes the empirical distribution: ${\mathrm{EIF}}_n(x)=\mathrm{IF}(x;T,F_n)$. For a smooth M-estimator it isIt requires a well-defined derivative and nonsingular denominator. An unsmoothed empirical quantile functional, for example, need not admit the population-style density derivative.

The actual one-observation sensitivity curve is a finite difference:It records contamination of size $1/(n+1)$ rather than an infinitesimal perturbation. For a smooth functional it approaches the population influence curve as the sample grows. A deletion counterpart uses $F_{n,-i}=(n{F}_n-{\delta }_{X_i})/(n-1)$, givingThe jackknife resampling pseudovalue $J_i=nT(F_n)-(n-1)T(F_{n,-i})$ is therefore approximately $T(F_n)+{\mathrm{EIF}}_n(X_i)$. The jackknife variance estimate $(n-1)n^{-1}{\sum }_i[T(F_{n,-i})-{\overline{T}}_{(-)}{]}^2$ corresponds asymptotically to $n^{-2}{\sum }_i{\mathrm{EIF}}_n(X_i{)}^2$ for centered empirical influences.

In linear regression, Cook's distance measures the fitted-value effect of deleting a case. For $p$ coefficients, residual $e_i$, leverage $h_{ii}$ and variance estimate ${\hat{\sigma }}^2$, its usual expression is $D_i=e_i^2{h}_{ii}/[p{\hat{\sigma }}^2(1-h_{ii}{)}^2]$. It illustrates that a case can be influential through both a large residual and high leverage. Finite perturbations and nonlinear refitting can differ materially from a first-order influence approximation; these diagnostics should not be conflated.

A curved exponential family restricts a higher-dimensional full canonical family to a lower-dimensional smooth parameter surface:The derivative matrix $J(\theta )=\partial \eta /\partial \theta$ has rank $q$. A genuinely curved surface is not merely an affine subfamily under a choice of natural coordinates. With $n$ iid observations the ambient sufficient statistic is $T_n={\sum }_{i}t(X_i)$, and the parameter score and information areThe second-derivative terms from $\eta (\theta )$ have zero expectation because the centered sufficient statistic has mean zero. Under regularity, the constrained MLE is still first-order asymptotically normal with this information. Curvature affects higher-order bias, likelihood geometry and efficiency; concavity of the ambient likelihood need not survive along the nonlinear parameter surface.

For example, $N(\mu ,{\mu }^2)$ with $\mu >0$ has $t(x)=(x,x^2)$ and natural parameters ${\eta }_1=1/\mu$, ${\eta }_2=-1/(2{\mu }^2)$, constrained by ${\eta }_2=-{\eta }_1^2/2$. It has one free parameter but generally still needs both $\sum X_i$ and $\sum X_i^2$ as sufficient statistics. To see why, a likelihood ratio between two samples is constant in $\mu$ only if their statistic differences satisfy $\Delta T_1/\mu -\Delta T_2/(2{\mu }^2)=\text{constant}$ for all positive $\mu$, forcing both differences to zero. Lower parameter dimension therefore does not generally reduce the minimal sufficient statistic to that dimension.