Statistics course of the University of Oxford (A9)

Source: released examination PDF, pages 191–194. Solutions follow the numbering of the 15 June 2023 Statistics paper.

Sort the observations into order statistics $x_{(1)}\le \cdots \le x_{(n)}$. Choose plotting positions $p_i=(i-1/2)/n$ and calculate the standard normal quantiles $z_i={\Phi }^{-1}(p_i)$. The normal Q-Q plot consists of the pairs $(z_i,x_{(i)})$.

If the normal distribution model is appropriate, the points should lie approximately on a straight line with positive slope, because its population quantile function isThe intercept is approximately $\mu$ and the slope approximately $\sigma$; neither needs to be known in advance. The reference need not be the identity line. One justification is the probability integral transform: for an independent normal statistical sample, $\Phi ((X_{(i)}-\mu )/\sigma )$ are uniform order statistics, whose typical values are near their plotting positions. The equality is approximate for a finite sample, particularly at the extremes. Systematic curvature, unusually heavy tails, or isolated extreme departures argue against the proposed distributional shape. A straight marginal Q-Q plot does not itself establish independence of observations.

Expanding the exponent, the log-likelihood function iswhere $C$ is independent of $\theta$. Its score function and second derivative areThe first is positive below $\bar{y}$ and negative above it, so the unique maximum-likelihood estimator isUsing $\mathbb{E}{\sum }_i{Y}_i=n\theta$, the one-observation Fisher information is $I(\theta )=-n^{-1}\mathbb{E}{\ell }^{\prime \prime }(\theta )={\theta }^{-3}$. The support does not depend on the interior parameter $\theta >0$, and the regular asymptotic normality of a maximum likelihood estimator givesThis is Inverse Gaussian mean estimation with known shape one. Equivalently, the Inverse Gaussian distribution has variance ${\theta }^3$, so the same result follows directly from the central limit theorem applied to its sample mean.

The estimated standard error is ${\hat{\theta }}^{3/2}/\sqrt{n}$. Using the central 95% standard normal quantiles $\pm 1.96$, a Wald confidence interval isConsistency of $\hat{\theta }$ and Slutsky's theorem justify replacing the unknown scale ${\theta }^{3/2}$ by its estimate. The coverage probability is asymptotic, not exact at finite $n$. Intersecting with the positive parameter space removes impossible values without changing whether the true positive parameter is covered.

Put $S={\sum }_i(x_i-\bar{x}{)}^2$ and $q=S/{\sigma }_0^2$. Under the null hypothesis, maximizing over the nuisance mean gives ${\hat{\mu }}_0=\bar{x}$. The unrestricted normal mean and variance maximum-likelihood estimators are $\hat{\mu }=\bar{x}$, $\widehat{{\sigma }^2}=S/n$. Thus, for $S>0$,The likelihood-ratio test statistic is consequentlyThe usual model assumes $n\ge 2$. The event $S=0$ has probability zero under a nondegenerate normal distribution; at that observation the unrestricted likelihood is unbounded as the variance tends to zero, and the ratio has limiting value zero. This is the likelihood-ratio test for a normal variance with unknown mean.

The full parameter space has dimension two and the null space dimension one. The regular Wilks theorem therefore gives $W\stackrel{d}{\to }{\chi }_1^2$ under $H_0$ as $n\to \infty$. If $F_1$ denotes the cumulative distribution function of the chi-squared distribution with one degree of freedom, the approximate size-$\alpha$ rejection region isThe corresponding approximate p-value isThe one degree of freedom here describes the large-sample likelihood-ratio test statistic; it is different from the exact residual degrees of freedom $n-1$ in the next part.

The derivative of the log ratio is $\frac{1}{2}(n/q-1)$, so $\lambda (q)$ increases from zero to one on $(0,n)$ and decreases to zero on $(n,\infty )$. Rejecting for a sufficiently small ratio therefore gives two tails with $0<q_1<n<q_2$.

Under $H_0$, Cochran's theorem gives the exact sampling distribution $Q=S/{\sigma }_0^2\sim {\chi }_{n-1}^2$, independently of the unknown mean. Writing $F_{n-1}$ for its cumulative distribution function, the two cutoffs are determined byThe first equation calibrates the exact size of a statistical test. The second is $\lambda (q_1)=\lambda (q_2)$, so both tails belong to one likelihood ratio cutoff. Hence the exact critical region is $\boxed{\{q\le q_1\}\cup \{q\ge q_2\}}$. Its two tail probabilities are not generally equal: equal-tailed chi-squared cutoffs give a different test.

For a critical region $C$, the size of a statistical test of the simple null hypothesis $P_0$ is $\alpha =P_0(\mathbf{X}\in C)$, the Type I error probability. Its statistical power against the simple alternative hypothesis $P_1$ is $P_1(\mathbf{X}\in C)$, the probability of correctly rejecting that null. For a randomized test function $\phi (\mathbf{x})\in [0,1]$, replace these by ${\mathbb{E}}_0\phi$ and ${\mathbb{E}}_1\phi$.

The Neyman-Pearson lemma states that a test which rejects where the joint likelihood ratio $L_1/L_0$ is largest is a most powerful test of its size: its power is at least that of any test with no larger size. More precisely, choose a threshold $k$ and reject if $L_1/L_0>k$, accept if it is smaller, and if necessary randomize on equality to attain the specified size.

For an independent and identically distributed statistical sample, takeThe $Y_i$ are independent and identically distributed random variables under either simple hypothesis, and the log joint ratio is their sum. Thus the nonrandomized threshold form can be writtenwith $c=-\log k$, using either strict or non-strict inequality when the boundary has probability zero. The usual finite-log expression assumes positive common support; zero densities can instead be handled by the extended likelihood ratio on the relevant supports. Continuous observations alone do not ensure a continuous likelihood-ratio statistic: ties may require the boundary randomization in the lemma. This qualification preserves the exact-size formulation.

The joint likelihood ratio in favor of the alternative isIt is increasing in the product, so the Neyman-Pearson lemma rejects for large product, equivalently small $S=-{\sum }_i\log x_i$. Under $H_0$, the transformation $Z_i=-\log X_i$ givesTherefore the $Z_i$ are independent exponential random variables of rate two, $S\sim \mathrm{Gamma}(n,\text{rate }2)$, and $4S\sim {\chi }_{2n}^2$. An exact size-$\alpha$ critical region isEquivalently, reject when the geometric mean satisfies $g\ge e^{-c_{\alpha }/n}$. The boundary distribution is continuous, so no randomization is needed here.

Under $H_0$, each $Z_i=-\log X_i$ has expected value $1/2$ and variance $1/4$. The central limit theorem therefore approximatesFor the supplied data this standardized value isUsing the provided approximation $\Phi (1.6)\approx 0.95$, the lower-tail 5% rule rejects when $S\le 12.5-1.6(2.5)=8.5$. The observed $6.25$ is in this region, with approximate p-value $\Phi (-2.5)\approx 0.005$. Reject $H_0:\theta =2$ at the 5% level in favor of the specified alternative $\theta =3$. The unusually large geometric mean is consistent with the direction of that alternative. This is evidence in this comparison, not a proof that the parameter is exactly three.

Given the observed data, a $100(1-\alpha )\%$ credible interval $[a,b]$ for the scalar parameter satisfiesThe probability refers to the posterior distribution, proportional to $\pi (\theta ){\prod }_{i}f(x_i\mid \theta )$ on the parameter space. It conditions on the observations; it is not the repeated-sampling coverage probability definition of a confidence interval.

An equal-tailed credible interval for a continuous posterior distribution has probability $\alpha /2$ below its lower endpoint and $\alpha /2$ above its upper endpoint. If $F$ is the posterior cumulative distribution function, the endpoints are $F^{-1}(\alpha /2)$ and $F^{-1}(1-\alpha /2)$.

A highest posterior density interval is a credible interval consisting of points of highest posterior density: every included point has density at least as high as every excluded point, apart from null-set and boundary choices. It is a posterior highest density region when that region is an interval. For a continuous unimodal density with two interior endpoints, their densities agree. A mode at a support boundary can instead force one endpoint to that boundary; a multimodal posterior can have a disconnected highest-density region rather than one interval. Equal tails and highest density need not give the same interval.

Writing $s={\sum }_i{x}_i$, the Poisson distribution likelihood is proportional to ${\theta }^s{e}^{-n\theta }$. Multiplying by the gamma distribution prior givesThis is Poisson-gamma conjugacy, with the rate convention made explicit:Its normalized density is $(\lambda +n{)}^{r+s}{\theta }^{r+s-1}{e}^{-(\lambda +n)\theta }/\Gamma (r+s)$ for $\theta >0$.

The posterior mean is a convex combination of the prior mean and the usual Poisson maximum-likelihood estimate:Both weights are positive and sum to one, so the mean lies between $r/\lambda$ and $\bar{x}$, strictly between them unless they coincide. If all counts are zero, $\bar{x}=0$ is the boundary estimate in the usual nonnegative Poisson parameter space; with a strictly positive parameter restriction it represents the likelihood supremum at that boundary. The displayed posterior-mean relation is valid in either interpretation.

Let $M=\max \{b,y_1,\dots ,y_n\}$. The continuous uniform distribution likelihood contributes ${\varphi }^{-n}{1}_{\{\varphi \ge {\max }_i{y}_i\}}$, while the Pareto distribution prior contributes ${\varphi }^{-a-1}{1}_{\{\varphi \ge b\}}$. Their product proves Uniform-Pareto conjugacy:Thus the posterior distribution is Pareto with shape $a+n$ and lower bound $M$. Its density is strictly decreasing on its support, so the highest posterior density interval begins at $M$. Since the upper-tail probability is $(M/u{)}^{a+n}$, the required upper endpoint satisfies $(M/u{)}^{a+n}=0.05$. The Pareto highest density interval is therefore

Write $m=r+s$ and $\beta =\lambda +n$. Conditional independence of the next count given $\theta$ allows us to average its Poisson distribution over the posterior distribution:This Poisson-gamma mixture is the posterior predictive distribution. In the stated negative binomial distribution convention its parameters areIts expected value $m/\beta$ equals the posterior mean of $\theta$, while its variance $m/\beta +m/{\beta }^2$ includes both conditional Poisson variation and uncertainty in $\theta$.

Source: released examination PDF, bundle pages 183–186. Solutions follow all three questions of the 18 June 2021 Statistics paper, including every lettered and Roman-numbered part. Each question carries 25 marks; the examination counted the best two answers.

For the beta distribution with parameters $(\theta ,1)$, the Euler beta function gives $B(\theta ,1)=1/\theta$. Its probability density function is therefore $\theta x^{\theta -1}$ on $(0,1)$. Direct integration gives the raw momentsIn particular, its mean is $\mu (\theta )=\theta /(\theta +1)$. Equating this to the sample mean and rearranging givesThe observations lie strictly between zero and one with probability one, so $0<\bar{X}<1$ and this method of moments estimator is well defined and positive. This is an instance of estimation of a beta shape parameter.

Using the raw moments from part (i), the variance of one observation isThe central limit theorem for the independent statistical sample givesFor $g(x)=x/(1-x)$, its derivative at the mean is $g^{\prime }(\mu )=(1-\mu {)}^{-2}=(\theta +1{)}^2$. The delta method therefore givesEquivalently, the large-sample normal approximation isHere $v$ denotes the leading asymptotic variance, rather than an assertion about an exact finite-sample variance.

Write $A=-{\sum }_i\log X_i>0$, with $\log$ the natural logarithm. The likelihood function and log-likelihood function areThe score function is ${\ell }^{\prime }(\theta )=n/\theta -A$ and the second derivative is ${\ell }^{\prime \prime }(\theta )=-n/{\theta }^2<0$. Thus the log-likelihood function is strictly concave, its stationary point is the unique global maximum, and the maximum-likelihood estimator isThe one-observation Fisher information is $I(\theta )=1/{\theta }^2$. The support of a probability distribution here is independent of $\theta >0$, so the regular asymptotic normality of a maximum likelihood estimator givesThere is also a direct check: $U_i=-\log X_i$ has an exponential distribution with rate $\theta$, since $\mathbb{P}(U_i>u)=\mathbb{P}(X_i<e^{-u})=e^{-\theta u}$ for $u>0$. Its mean and variance are $1/\theta$ and $1/{\theta }^2$. Apply the central limit theorem to $A/n$ and the delta method to $h(u)=1/u$ to obtain the same result.

The specified asymptotic relative efficiency isFor every fixed $\theta >0$ this lies strictly between zero and one. Prefer $T_2$ on grounds of asymptotic precision: it has the smaller leading asymptotic variance and attains the inverse Fisher information. Both are consistent estimators, by the law of large numbers and the continuous mapping theorem applied respectively to $\bar{X}$ and $A/n$. The efficiency approaches zero as $\theta \downarrow 0$ and one as $\theta \to \infty$.

This preference is an asymptotic comparison. In particular, it does not assert that $T_2$ is an unbiased estimator: since $A$ has a gamma distribution of shape $n$ and rate $\theta$, $\mathbb{E}{T}_2=n\theta /(n-1)$ for $n>1$.

The standard error suggested by part (iii) is $\theta /\sqrt{n}$. Substituting the consistent estimator $T_2$ gives $T_2/\sqrt{n}$; Slutsky's theorem givesUsing the central standard normal quantiles $\pm 1.96$, an approximate 95% Wald confidence interval isIts coverage probability approaches 0.95 for each fixed $\theta >0$ as $n\to \infty$. The intersection enforces positivity and does not change whether the true positive statistical parameter is covered. The value 1.64 would give the wrong central two-sided coverage probability.

Put $A=-{\sum }_i\log x_i>0$, $B=-{\sum }_j\log y_j>0$, and $N=n+m$. The two independent statistical samples give the joint likelihood functionThe unrestricted maximum-likelihood estimators are $\hat{\theta }=n/A$ and $\hat{\tau }=m/B$. Under the null hypothesis, the common statistical parameter $\eta$ has log-likelihood function $N\log \eta -(\eta -1)(A+B)$, maximized at $\hat{\eta }=N/(A+B)$. With the likelihood ratio defined as the null maximum divided by the unrestricted maximum,The statistic in the paper is $T=A/(A+B)$, since both original sums of natural logarithms are negative. HenceIt has maximum one at $T=n/N$, corresponding to equal fitted rates. This is the likelihood-ratio test for equality of beta shape parameters.

A likelihood-ratio test rejects for small $\lambda$. The derivative of the natural logarithm of its variable factor isThus $t^n(1-t{)}^m$ increases up to $n/(n+m)$ and decreases thereafter, tending to zero at both endpoints. Every nontrivial lower level set of $\lambda$ therefore consists of two tails. For a prescribed test size $0<\alpha <1$, the exact critical region isWriting $F_{n,m}$ for the cumulative distribution function of the beta distribution with parameters $(n,m)$, the required two equations areThe first makes both cutoffs have the same likelihood ratio; the second calibrates the null hypothesis rejection probability. Equivalently, the second left-hand side isAs the common likelihood ratio threshold increases from zero to one, this probability increases continuously and strictly from zero to one; hence the equations determine unique cutoffs. The two tail probabilities need not each equal $\alpha /2$ when $n\ne m$. Exact calibration holds for every common rate because the given null distribution of $T$ is independent of that rate. Boundary equalities have zero probability.

False. A test size constrains the probability of rejection conditional on the null hypothesis being true:for an exact size-$\alpha$ test of a simple hypothesis. It does not assign a posterior probability to that statistical hypothesis after observing nonrejection. In the frequentist interpretation, the statistical parameter is fixed and the data are random. Nonrejection can occur even when the alternative hypothesis is true; for example, it is particularly likely for an alternative against which the test has low statistical power. A posterior probability would require a prior distribution and a Bayesian model.

True, with the test's rejection rule fixed. If its rejection region in the space of the test statistic is $C$, its statistical power against the specified simple hypothesis $H_1$ isThus the probability distribution of $T$ under the alternative hypothesis determines the statistical power of this test. The null distribution is used to choose a rule with the desired test size, but statistical power is evaluated under the alternative hypothesis. Different rejection rules can have different statistical power even for the same probability distribution of $T$.

False. For the test associated with a p-value, reject at significance level $\alpha$ when $p\le \alpha$. Here $0.1>0.05$, so do not reject at level 0.05. The p-value is neither the posterior probability of the null hypothesis nor the probability of a Type I error conditional on these observed data.

Let $S={\sum }_i{X}_i$, which has a binomial distribution with parameters $(n,p)$. For $0<p_1<1/2$, the alternative-to-null likelihood ratio of the independent Bernoulli trials isSince $p_1/(1-p_1)<1$, this is strictly decreasing in $S$. A threshold on large $R$ is therefore a lower-tail likelihood-ratio test, and a threshold between $R(2)$ and $R(1)$ gives the stated critical region $S<2$. The Neyman-Pearson lemma also proves it is a most powerful test at its attained test size.

Using the binomial probability mass, its test size and statistical power are respectivelyThe associated power function is $\pi (p)=(1-p{)}^n+np(1-p{)}^{n-1}$. These formulas hold for positive integer $n$; when $n=1$ the test always rejects and has size one. If the boundary value $p_1=0$ is admitted, all alternative probability is at $S=0$; the same test still has statistical power one and can be represented using a threshold with ties in the likelihood ratio.

For any fixed $p>1/4$, the alternative-to-null likelihood ratio iswhich is strictly increasing in $S$ for $1/4<p<1$. Consequently the Neyman-Pearson lemma selects the same upper-tail ordering for every such alternative hypothesis. At the null hypothesis value,Thereforeis a uniformly most powerful test of the required exact test size. Its power function is $5p^4(1-p)+p^5=5p^4-4p^5$. At $p=1$ its statistical power is one, so including that boundary alternative preserves the conclusion.

Condition on the total of 400 complaints. Assume each complaint independently has the same probability $p$ of being received on Monday; in particular, repeated complaints from one incident must not create material dependence. The Monday count $S$ then has a binomial distribution with parameters $(400,p)$. Test $H_0:p=0.2$ against $H_1:p>0.2$, rejecting for large $S$ as suggested by the monotone likelihood ratio.

Under the null hypothesis, ${\mathbb{E}}_{0}S=80$ and ${\mathrm{Var}}_0(S)=400(0.2)(0.8)=64$. Both expected successes and expected failures are large, so the central limit theorem gives a suitable normal approximation:Using the paper's supplied rounded table, the upper-tail p-value is approximatelyA more accurate evaluation of the standard normal distribution cumulative distribution function gives about 0.0062. For example, choose significance level 0.05 before examining the data; the supplied upper 5% standard normal quantile is approximately 1.6, and $2.5>1.6$. Reject $H_0$ at 5%, and also at 1%, using this approximation. The observed binomial proportion is $100/400=0.25$, providing evidence that the Monday probability exceeds 0.2 under the sampling assumptions.

If applying a continuity correction, approximate ${\mathbb{P}}_0(S\ge 100)$ by $1-\Phi ((99.5-80)/8)=1-\Phi (2.4375)\approx 0.0074$; this gives the same 5% and 1% conclusions. The uncorrected value 0.005 is approximate, so a decision exactly at the 0.5% boundary is not warranted by the supplied rounded table.

Here the gamma distribution uses a rate statistical parameter $\theta$, with known shape $\lambda$. Write $S={\sum }_i{X}_i$ and $a=n\lambda$. For each fixed ${\theta }_1>{\theta }_0$, the alternative-to-null likelihood ratio isstrictly decreasing in $S$. The Neyman-Pearson lemma therefore gives the same lower-tail critical region $S<c_{\alpha }$ for every larger rate. Since $S$ has a continuous gamma distribution under the null hypothesis, there is a unique positive $c_{\alpha }$ satisfyingThus reject when $S<c_{\alpha }$ is a uniformly most powerful test of exact test size $\alpha$. If $G_a$ denotes the unit-rate gamma distribution cumulative distribution function with shape $a$, thenThis is the uniformly most powerful test for a gamma rate.

When $\lambda =1/n$, the sum has shape one and hence an exponential distribution with rate $\theta$. ThereforeIn particular $\pi ({\theta }_0)=\alpha$, and the power function is strictly increasing, exceeding $\alpha$ for $\theta >{\theta }_0$ and tending to one as $\theta \to \infty$.

Set $S={\sum }_i{x}_i$ and $N=nm$. Conditional independence of the binomial distribution observations gives the likelihood functionThe prior distribution is the beta distribution with parameters $(1,\beta )$. By Bayes' theorem, multiplying the prior density by the likelihood function gives a posterior density proportional toRecognizing and normalizing this beta distribution givesfor $0<\theta <1$. This is Beta-binomial conjugacy; the original binomial coefficients cancel in the normalization because they are independent of $\theta$.

The prior mean, posterior mean, and maximum-likelihood estimate areFor $0<S<N$, the last follows by maximizing $S\log \theta +(N-S)\log (1-\theta )$. At $S=0$ or $S=N$, use the usual binomial likelihood convention that admits the endpoints $0$ and $1$ as maximum-likelihood estimates; if the parameter domain is strictly $(0,1)$, these are boundary suprema rather than attained maxima. The posterior mean remains interior in all cases.