Past exam of the statistics course of the University of Oxford / 2021 (A9, A13765W1)

Source: released examination PDF, bundle pages 183–186. Solutions follow all three questions of the 18 June 2021 Statistics paper, including every lettered and Roman-numbered part. Each question carries 25 marks; the examination counted the best two answers.

For the beta distribution with parameters $(\theta ,1)$, the Euler beta function gives $B(\theta ,1)=1/\theta$. Its probability density function is therefore $\theta x^{\theta -1}$ on $(0,1)$. Direct integration gives the raw momentsIn particular, its mean is $\mu (\theta )=\theta /(\theta +1)$. Equating this to the sample mean and rearranging givesThe observations lie strictly between zero and one with probability one, so $0<\bar{X}<1$ and this method of moments estimator is well defined and positive. This is an instance of estimation of a beta shape parameter.

Using the raw moments from part (i), the variance of one observation isThe central limit theorem for the independent statistical sample givesFor $g(x)=x/(1-x)$, its derivative at the mean is $g^{\prime }(\mu )=(1-\mu {)}^{-2}=(\theta +1{)}^2$. The delta method therefore givesEquivalently, the large-sample normal approximation isHere $v$ denotes the leading asymptotic variance, rather than an assertion about an exact finite-sample variance.

Write $A=-{\sum }_i\log X_i>0$, with $\log$ the natural logarithm. The likelihood function and log-likelihood function areThe score function is ${\ell }^{\prime }(\theta )=n/\theta -A$ and the second derivative is ${\ell }^{\prime \prime }(\theta )=-n/{\theta }^2<0$. Thus the log-likelihood function is strictly concave, its stationary point is the unique global maximum, and the maximum-likelihood estimator isThe one-observation Fisher information is $I(\theta )=1/{\theta }^2$. The support of a probability distribution here is independent of $\theta >0$, so the regular asymptotic normality of a maximum likelihood estimator givesThere is also a direct check: $U_i=-\log X_i$ has an exponential distribution with rate $\theta$, since $\mathbb{P}(U_i>u)=\mathbb{P}(X_i<e^{-u})=e^{-\theta u}$ for $u>0$. Its mean and variance are $1/\theta$ and $1/{\theta }^2$. Apply the central limit theorem to $A/n$ and the delta method to $h(u)=1/u$ to obtain the same result.

The specified asymptotic relative efficiency isFor every fixed $\theta >0$ this lies strictly between zero and one. Prefer $T_2$ on grounds of asymptotic precision: it has the smaller leading asymptotic variance and attains the inverse Fisher information. Both are consistent estimators, by the law of large numbers and the continuous mapping theorem applied respectively to $\bar{X}$ and $A/n$. The efficiency approaches zero as $\theta \downarrow 0$ and one as $\theta \to \infty$.

This preference is an asymptotic comparison. In particular, it does not assert that $T_2$ is an unbiased estimator: since $A$ has a gamma distribution of shape $n$ and rate $\theta$, $\mathbb{E}{T}_2=n\theta /(n-1)$ for $n>1$.

The standard error suggested by part (iii) is $\theta /\sqrt{n}$. Substituting the consistent estimator $T_2$ gives $T_2/\sqrt{n}$; Slutsky's theorem givesUsing the central standard normal quantiles $\pm 1.96$, an approximate 95% Wald confidence interval isIts coverage probability approaches 0.95 for each fixed $\theta >0$ as $n\to \infty$. The intersection enforces positivity and does not change whether the true positive statistical parameter is covered. The value 1.64 would give the wrong central two-sided coverage probability.

Put $A=-{\sum }_i\log x_i>0$, $B=-{\sum }_j\log y_j>0$, and $N=n+m$. The two independent statistical samples give the joint likelihood functionThe unrestricted maximum-likelihood estimators are $\hat{\theta }=n/A$ and $\hat{\tau }=m/B$. Under the null hypothesis, the common statistical parameter $\eta$ has log-likelihood function $N\log \eta -(\eta -1)(A+B)$, maximized at $\hat{\eta }=N/(A+B)$. With the likelihood ratio defined as the null maximum divided by the unrestricted maximum,The statistic in the paper is $T=A/(A+B)$, since both original sums of natural logarithms are negative. HenceIt has maximum one at $T=n/N$, corresponding to equal fitted rates. This is the likelihood-ratio test for equality of beta shape parameters.

A likelihood-ratio test rejects for small $\lambda$. The derivative of the natural logarithm of its variable factor isThus $t^n(1-t{)}^m$ increases up to $n/(n+m)$ and decreases thereafter, tending to zero at both endpoints. Every nontrivial lower level set of $\lambda$ therefore consists of two tails. For a prescribed test size $0<\alpha <1$, the exact critical region isWriting $F_{n,m}$ for the cumulative distribution function of the beta distribution with parameters $(n,m)$, the required two equations areThe first makes both cutoffs have the same likelihood ratio; the second calibrates the null hypothesis rejection probability. Equivalently, the second left-hand side isAs the common likelihood ratio threshold increases from zero to one, this probability increases continuously and strictly from zero to one; hence the equations determine unique cutoffs. The two tail probabilities need not each equal $\alpha /2$ when $n\ne m$. Exact calibration holds for every common rate because the given null distribution of $T$ is independent of that rate. Boundary equalities have zero probability.

False. A test size constrains the probability of rejection conditional on the null hypothesis being true:for an exact size-$\alpha$ test of a simple hypothesis. It does not assign a posterior probability to that statistical hypothesis after observing nonrejection. In the frequentist interpretation, the statistical parameter is fixed and the data are random. Nonrejection can occur even when the alternative hypothesis is true; for example, it is particularly likely for an alternative against which the test has low statistical power. A posterior probability would require a prior distribution and a Bayesian model.

True, with the test's rejection rule fixed. If its rejection region in the space of the test statistic is $C$, its statistical power against the specified simple hypothesis $H_1$ isThus the probability distribution of $T$ under the alternative hypothesis determines the statistical power of this test. The null distribution is used to choose a rule with the desired test size, but statistical power is evaluated under the alternative hypothesis. Different rejection rules can have different statistical power even for the same probability distribution of $T$.

False. For the test associated with a p-value, reject at significance level $\alpha$ when $p\le \alpha$. Here $0.1>0.05$, so do not reject at level 0.05. The p-value is neither the posterior probability of the null hypothesis nor the probability of a Type I error conditional on these observed data.

Let $S={\sum }_i{X}_i$, which has a binomial distribution with parameters $(n,p)$. For $0<p_1<1/2$, the alternative-to-null likelihood ratio of the independent Bernoulli trials isSince $p_1/(1-p_1)<1$, this is strictly decreasing in $S$. A threshold on large $R$ is therefore a lower-tail likelihood-ratio test, and a threshold between $R(2)$ and $R(1)$ gives the stated critical region $S<2$. The Neyman-Pearson lemma also proves it is a most powerful test at its attained test size.

Using the binomial probability mass, its test size and statistical power are respectivelyThe associated power function is $\pi (p)=(1-p{)}^n+np(1-p{)}^{n-1}$. These formulas hold for positive integer $n$; when $n=1$ the test always rejects and has size one. If the boundary value $p_1=0$ is admitted, all alternative probability is at $S=0$; the same test still has statistical power one and can be represented using a threshold with ties in the likelihood ratio.

For any fixed $p>1/4$, the alternative-to-null likelihood ratio iswhich is strictly increasing in $S$ for $1/4<p<1$. Consequently the Neyman-Pearson lemma selects the same upper-tail ordering for every such alternative hypothesis. At the null hypothesis value,Thereforeis a uniformly most powerful test of the required exact test size. Its power function is $5p^4(1-p)+p^5=5p^4-4p^5$. At $p=1$ its statistical power is one, so including that boundary alternative preserves the conclusion.

Condition on the total of 400 complaints. Assume each complaint independently has the same probability $p$ of being received on Monday; in particular, repeated complaints from one incident must not create material dependence. The Monday count $S$ then has a binomial distribution with parameters $(400,p)$. Test $H_0:p=0.2$ against $H_1:p>0.2$, rejecting for large $S$ as suggested by the monotone likelihood ratio.

Under the null hypothesis, ${\mathbb{E}}_{0}S=80$ and ${\mathrm{Var}}_0(S)=400(0.2)(0.8)=64$. Both expected successes and expected failures are large, so the central limit theorem gives a suitable normal approximation:Using the paper's supplied rounded table, the upper-tail p-value is approximatelyA more accurate evaluation of the standard normal distribution cumulative distribution function gives about 0.0062. For example, choose significance level 0.05 before examining the data; the supplied upper 5% standard normal quantile is approximately 1.6, and $2.5>1.6$. Reject $H_0$ at 5%, and also at 1%, using this approximation. The observed binomial proportion is $100/400=0.25$, providing evidence that the Monday probability exceeds 0.2 under the sampling assumptions.

If applying a continuity correction, approximate ${\mathbb{P}}_0(S\ge 100)$ by $1-\Phi ((99.5-80)/8)=1-\Phi (2.4375)\approx 0.0074$; this gives the same 5% and 1% conclusions. The uncorrected value 0.005 is approximate, so a decision exactly at the 0.5% boundary is not warranted by the supplied rounded table.

Here the gamma distribution uses a rate statistical parameter $\theta$, with known shape $\lambda$. Write $S={\sum }_i{X}_i$ and $a=n\lambda$. For each fixed ${\theta }_1>{\theta }_0$, the alternative-to-null likelihood ratio isstrictly decreasing in $S$. The Neyman-Pearson lemma therefore gives the same lower-tail critical region $S<c_{\alpha }$ for every larger rate. Since $S$ has a continuous gamma distribution under the null hypothesis, there is a unique positive $c_{\alpha }$ satisfyingThus reject when $S<c_{\alpha }$ is a uniformly most powerful test of exact test size $\alpha$. If $G_a$ denotes the unit-rate gamma distribution cumulative distribution function with shape $a$, thenThis is the uniformly most powerful test for a gamma rate.

When $\lambda =1/n$, the sum has shape one and hence an exponential distribution with rate $\theta$. ThereforeIn particular $\pi ({\theta }_0)=\alpha$, and the power function is strictly increasing, exceeding $\alpha$ for $\theta >{\theta }_0$ and tending to one as $\theta \to \infty$.

Set $S={\sum }_i{x}_i$ and $N=nm$. Conditional independence of the binomial distribution observations gives the likelihood functionThe prior distribution is the beta distribution with parameters $(1,\beta )$. By Bayes' theorem, multiplying the prior density by the likelihood function gives a posterior density proportional toRecognizing and normalizing this beta distribution givesfor $0<\theta <1$. This is Beta-binomial conjugacy; the original binomial coefficients cancel in the normalization because they are independent of $\theta$.

The prior mean, posterior mean, and maximum-likelihood estimate areFor $0<S<N$, the last follows by maximizing $S\log \theta +(N-S)\log (1-\theta )$. At $S=0$ or $S=N$, use the usual binomial likelihood convention that admits the endpoints $0$ and $1$ as maximum-likelihood estimates; if the parameter domain is strictly $(0,1)$, these are boundary suprema rather than attained maxima. The posterior mean remains interior in all cases.

TakinggivesSince $N=nm>0$ and $\beta +1>0$, $0<\lambda <1$. Thus the posterior mean is a convex combination of the prior mean and observed binomial proportion, with larger prior weight when $\beta +1$ is large relative to $N$.

The prior mean $1/(\beta +1)$ decreases as $\beta$ increases. More strongly, its cumulative distribution function and upper-tail probability areFor every fixed $t$, that upper-tail probability decreases with $\beta$. The statistician increasingly favors smaller values of $\theta$; as $\beta \to \infty$, the prior distribution concentrates near zero. For $\beta >1$, its variance $\beta /[(\beta +1{)}^2(\beta +2)]$ also decreases. This is concentration of a beta prior near zero.

With the data fixed, the derivatives areThus the prior weight increases and the posterior mean decreases. Their limits are $\lambda \to 1$ and ${\tilde{\theta }}_1\to 0$. Both the changing prior mean and the increasing weight contribute to the change in the posterior mean.

Let $S_{xx}={\sum }_i{x}_i^2$, $S_{xy}={\sum }_i{x}_i{y}_i$, and $\bar{y}=n^{-1}{\sum }_i{y}_i$. The independent observations in the normal linear model giveThe centered design condition ${\sum }_i{x}_i=0$ removes the $\alpha \beta$ term:The stated prior density factors into independent normal distributions for the regression intercept $\alpha$ and regression coefficient $\beta$. Multiplication by the likelihood function therefore leaves separate quadratic expressions in the two coefficients. Define the positive posterior variancesand the posterior meansBy completing the square, the joint posterior density isEquivalently,This is Gaussian conjugacy for a normal linear model. Each posterior precision parameter is the sum of its data and prior precision parameters. The formulas remain valid when $S_{xx}=0$: all $x_i$ are then zero, so the posterior distribution of $\beta$ equals its prior distribution.

Put $M_0=\alpha +x_0\beta$. As a linear combination of independent normal random variables under the posterior distribution,The shortest 95% credible interval isHere $\Phi$ is the standard normal distribution cumulative distribution function. To justify shortest length, consider all intervals of a fixed length $d$. If $f$ is this normal distribution posterior density, the interval probability ${\int }_a^{a+d}f(t)dt$ has derivative $f(a+d)-f(a)$ with respect to $a$. The normal distribution is symmetric and its density strictly decreases with distance from its mean, so this probability is uniquely maximized when the midpoint is $m_0$. The centered interval can therefore attain 95% posterior probability with the least length. It is both an equal-tailed credible interval and a highest posterior density interval.

The target is the experiment's mean at $x_0$. A posterior predictive distribution for a new noisy observation would instead have variance $v_0+{\sigma }^2$; that extra observation variance does not belong in the requested interval.

An unknown variance requires a joint prior distribution for $(\alpha ,\beta ,{\sigma }^2)$. The analysis is not determined until a prior for the unknown variance, and its relation to the coefficients, is specified. A maximum-likelihood estimate substituted for ${\sigma }^2$ does not by itself account for its posterior uncertainty.

For example, retain the independent coefficient prior distributions from part (b) and independently assign $s={\sigma }^2$ an inverse-gamma distribution with shape $a_0>0$ and scale $b_0>0$, whose probability density function is proportional to $s^{-a_0-1}{e}^{-b_0/s}$. Then the joint posterior density is proportional toConditional on $s$, the two coefficient posterior distributions are still the independent normal distributions in part (b), with $s$ substituted for ${\sigma }^2$. The other full conditional distribution isThe marginal coefficient posterior distribution requires marginalization over $s$; it is generally nonnormal and the two coefficients need not remain independent. In particular, the posterior cumulative distribution function of the requested mean becomesIts shortest 95% credible interval must be found from this marginal posterior distribution, for example by numerical integration or posterior simulation. This is normal regression with independent coefficient and variance priors.

A different, convenient choice is a normal-inverse-gamma prior, in which the conditional coefficient prior covariance matrix is proportional to ${\sigma }^2$. That choice gives a multivariate Student's t-distribution after marginalization, so the interval uses its univariate Student's t-distribution quantiles. It changes the coefficient prior specification; a Student interval does not follow automatically from retaining the fixed-${\tau }_0,{\tau }_1$ priors in part (b).