Past exam of the statistics course of the University of Oxford / 2023 (A9, A13765W1)

Source: released examination PDF, pages 191–194. Solutions follow the numbering of the 15 June 2023 Statistics paper.

Sort the observations into order statistics $x_{(1)}\le \cdots \le x_{(n)}$. Choose plotting positions $p_i=(i-1/2)/n$ and calculate the standard normal quantiles $z_i={\Phi }^{-1}(p_i)$. The normal Q-Q plot consists of the pairs $(z_i,x_{(i)})$.

If the normal distribution model is appropriate, the points should lie approximately on a straight line with positive slope, because its population quantile function isThe intercept is approximately $\mu$ and the slope approximately $\sigma$; neither needs to be known in advance. The reference need not be the identity line. One justification is the probability integral transform: for an independent normal statistical sample, $\Phi ((X_{(i)}-\mu )/\sigma )$ are uniform order statistics, whose typical values are near their plotting positions. The equality is approximate for a finite sample, particularly at the extremes. Systematic curvature, unusually heavy tails, or isolated extreme departures argue against the proposed distributional shape. A straight marginal Q-Q plot does not itself establish independence of observations.

Expanding the exponent, the log-likelihood function iswhere $C$ is independent of $\theta$. Its score function and second derivative areThe first is positive below $\bar{y}$ and negative above it, so the unique maximum-likelihood estimator isUsing $\mathbb{E}{\sum }_i{Y}_i=n\theta$, the one-observation Fisher information is $I(\theta )=-n^{-1}\mathbb{E}{\ell }^{\prime \prime }(\theta )={\theta }^{-3}$. The support does not depend on the interior parameter $\theta >0$, and the regular asymptotic normality of a maximum likelihood estimator givesThis is Inverse Gaussian mean estimation with known shape one. Equivalently, the Inverse Gaussian distribution has variance ${\theta }^3$, so the same result follows directly from the central limit theorem applied to its sample mean.

The estimated standard error is ${\hat{\theta }}^{3/2}/\sqrt{n}$. Using the central 95% standard normal quantiles $\pm 1.96$, a Wald confidence interval isConsistency of $\hat{\theta }$ and Slutsky's theorem justify replacing the unknown scale ${\theta }^{3/2}$ by its estimate. The coverage probability is asymptotic, not exact at finite $n$. Intersecting with the positive parameter space removes impossible values without changing whether the true positive parameter is covered.

Put $S={\sum }_i(x_i-\bar{x}{)}^2$ and $q=S/{\sigma }_0^2$. Under the null hypothesis, maximizing over the nuisance mean gives ${\hat{\mu }}_0=\bar{x}$. The unrestricted normal mean and variance maximum-likelihood estimators are $\hat{\mu }=\bar{x}$, $\widehat{{\sigma }^2}=S/n$. Thus, for $S>0$,The likelihood-ratio test statistic is consequentlyThe usual model assumes $n\ge 2$. The event $S=0$ has probability zero under a nondegenerate normal distribution; at that observation the unrestricted likelihood is unbounded as the variance tends to zero, and the ratio has limiting value zero. This is the likelihood-ratio test for a normal variance with unknown mean.

The full parameter space has dimension two and the null space dimension one. The regular Wilks theorem therefore gives $W\stackrel{d}{\to }{\chi }_1^2$ under $H_0$ as $n\to \infty$. If $F_1$ denotes the cumulative distribution function of the chi-squared distribution with one degree of freedom, the approximate size-$\alpha$ rejection region isThe corresponding approximate p-value isThe one degree of freedom here describes the large-sample likelihood-ratio test statistic; it is different from the exact residual degrees of freedom $n-1$ in the next part.

The derivative of the log ratio is $\frac{1}{2}(n/q-1)$, so $\lambda (q)$ increases from zero to one on $(0,n)$ and decreases to zero on $(n,\infty )$. Rejecting for a sufficiently small ratio therefore gives two tails with $0<q_1<n<q_2$.

Under $H_0$, Cochran's theorem gives the exact sampling distribution $Q=S/{\sigma }_0^2\sim {\chi }_{n-1}^2$, independently of the unknown mean. Writing $F_{n-1}$ for its cumulative distribution function, the two cutoffs are determined byThe first equation calibrates the exact size of a statistical test. The second is $\lambda (q_1)=\lambda (q_2)$, so both tails belong to one likelihood ratio cutoff. Hence the exact critical region is $\boxed{\{q\le q_1\}\cup \{q\ge q_2\}}$. Its two tail probabilities are not generally equal: equal-tailed chi-squared cutoffs give a different test.

For a critical region $C$, the size of a statistical test of the simple null hypothesis $P_0$ is $\alpha =P_0(\mathbf{X}\in C)$, the Type I error probability. Its statistical power against the simple alternative hypothesis $P_1$ is $P_1(\mathbf{X}\in C)$, the probability of correctly rejecting that null. For a randomized test function $\phi (\mathbf{x})\in [0,1]$, replace these by ${\mathbb{E}}_0\phi$ and ${\mathbb{E}}_1\phi$.

The Neyman-Pearson lemma states that a test which rejects where the joint likelihood ratio $L_1/L_0$ is largest is a most powerful test of its size: its power is at least that of any test with no larger size. More precisely, choose a threshold $k$ and reject if $L_1/L_0>k$, accept if it is smaller, and if necessary randomize on equality to attain the specified size.

For an independent and identically distributed statistical sample, takeThe $Y_i$ are independent and identically distributed random variables under either simple hypothesis, and the log joint ratio is their sum. Thus the nonrandomized threshold form can be writtenwith $c=-\log k$, using either strict or non-strict inequality when the boundary has probability zero. The usual finite-log expression assumes positive common support; zero densities can instead be handled by the extended likelihood ratio on the relevant supports. Continuous observations alone do not ensure a continuous likelihood-ratio statistic: ties may require the boundary randomization in the lemma. This qualification preserves the exact-size formulation.

The joint likelihood ratio in favor of the alternative isIt is increasing in the product, so the Neyman-Pearson lemma rejects for large product, equivalently small $S=-{\sum }_i\log x_i$. Under $H_0$, the transformation $Z_i=-\log X_i$ givesTherefore the $Z_i$ are independent exponential random variables of rate two, $S\sim \mathrm{Gamma}(n,\text{rate }2)$, and $4S\sim {\chi }_{2n}^2$. An exact size-$\alpha$ critical region isEquivalently, reject when the geometric mean satisfies $g\ge e^{-c_{\alpha }/n}$. The boundary distribution is continuous, so no randomization is needed here.

Under $H_0$, each $Z_i=-\log X_i$ has expected value $1/2$ and variance $1/4$. The central limit theorem therefore approximatesFor the supplied data this standardized value isUsing the provided approximation $\Phi (1.6)\approx 0.95$, the lower-tail 5% rule rejects when $S\le 12.5-1.6(2.5)=8.5$. The observed $6.25$ is in this region, with approximate p-value $\Phi (-2.5)\approx 0.005$. Reject $H_0:\theta =2$ at the 5% level in favor of the specified alternative $\theta =3$. The unusually large geometric mean is consistent with the direction of that alternative. This is evidence in this comparison, not a proof that the parameter is exactly three.

Given the observed data, a $100(1-\alpha )\%$ credible interval $[a,b]$ for the scalar parameter satisfiesThe probability refers to the posterior distribution, proportional to $\pi (\theta ){\prod }_{i}f(x_i\mid \theta )$ on the parameter space. It conditions on the observations; it is not the repeated-sampling coverage probability definition of a confidence interval.

An equal-tailed credible interval for a continuous posterior distribution has probability $\alpha /2$ below its lower endpoint and $\alpha /2$ above its upper endpoint. If $F$ is the posterior cumulative distribution function, the endpoints are $F^{-1}(\alpha /2)$ and $F^{-1}(1-\alpha /2)$.

A highest posterior density interval is a credible interval consisting of points of highest posterior density: every included point has density at least as high as every excluded point, apart from null-set and boundary choices. It is a posterior highest density region when that region is an interval. For a continuous unimodal density with two interior endpoints, their densities agree. A mode at a support boundary can instead force one endpoint to that boundary; a multimodal posterior can have a disconnected highest-density region rather than one interval. Equal tails and highest density need not give the same interval.

Writing $s={\sum }_i{x}_i$, the Poisson distribution likelihood is proportional to ${\theta }^s{e}^{-n\theta }$. Multiplying by the gamma distribution prior givesThis is Poisson-gamma conjugacy, with the rate convention made explicit:Its normalized density is $(\lambda +n{)}^{r+s}{\theta }^{r+s-1}{e}^{-(\lambda +n)\theta }/\Gamma (r+s)$ for $\theta >0$.

The posterior mean is a convex combination of the prior mean and the usual Poisson maximum-likelihood estimate:Both weights are positive and sum to one, so the mean lies between $r/\lambda$ and $\bar{x}$, strictly between them unless they coincide. If all counts are zero, $\bar{x}=0$ is the boundary estimate in the usual nonnegative Poisson parameter space; with a strictly positive parameter restriction it represents the likelihood supremum at that boundary. The displayed posterior-mean relation is valid in either interpretation.

Let $M=\max \{b,y_1,\dots ,y_n\}$. The continuous uniform distribution likelihood contributes ${\varphi }^{-n}{1}_{\{\varphi \ge {\max }_i{y}_i\}}$, while the Pareto distribution prior contributes ${\varphi }^{-a-1}{1}_{\{\varphi \ge b\}}$. Their product proves Uniform-Pareto conjugacy:Thus the posterior distribution is Pareto with shape $a+n$ and lower bound $M$. Its density is strictly decreasing on its support, so the highest posterior density interval begins at $M$. Since the upper-tail probability is $(M/u{)}^{a+n}$, the required upper endpoint satisfies $(M/u{)}^{a+n}=0.05$. The Pareto highest density interval is therefore

Write $m=r+s$ and $\beta =\lambda +n$. Conditional independence of the next count given $\theta$ allows us to average its Poisson distribution over the posterior distribution:This Poisson-gamma mixture is the posterior predictive distribution. In the stated negative binomial distribution convention its parameters areIts expected value $m/\beta$ equals the posterior mean of $\theta$, while its variance $m/\beta +m/{\beta }^2$ includes both conditional Poisson variation and uncertainty in $\theta$.