Past exam of the mathematics course of the University of Cambridge / 2014 / iii / Paper 35 / 3

At zero a centered normal distribution with standard deviation $\sigma$ has density $1/(\sqrt{2\pi }\sigma )$. The two prior standard deviations are $1/\sqrt{n_0}$ and $c/\sqrt{n_0}$, givingThe wider prior distribution has a lower density at its center.

Let $q_0=n_0$, $q_1=n_0/c^2$ and $y=\overline{y}$. These are the two prior precision parameters. Completing the square in $n(\beta -y{)}^2+q_i{\beta }^2$ gives the Normal-normal conjugacy updateThus the posterior mean is a precision-weighted average of the observed mean and the prior center zero. The posterior variance is the reciprocal of the total precision.

Write $\overline{Y}=\beta +\epsilon$, with independent $\epsilon \sim N(0,1/n)$ and $\beta \mid H_i\sim N(0,1/q_i)$. The convolution of independent random variables is again a normal distribution, so the prior predictive laws areThese are predictive distributions before observing $y$, hence the Bayesian model evidence for the observed mean. The residual information in the original observations is common to both models and cancels in their Bayes factor.

Divide the two normal distribution predictive densities to obtain the Bayes factorAt $y=0$ this becomesThe wider prior distribution spreads its predictive mass over more possible means, giving the narrower model more Bayesian model evidence for observations very near zero. Away from zero, the exponential term opposes that factor.

Under the point hypothesis $\beta =0$, the test statistic $\sqrt{n}\overline{Y}$ has a standard normal distribution. The observed statistic is three, giving a two-sided p-value $2[1-\Phi (3)]\simeq 0.00270$. This is conventionally strong evidence against that point hypothesis. The narrow model $H_0$ is a continuous prior distribution around zero, rather than a point hypothesis.

For $n_0/n=100$ and $c=100$, Normal-normal conjugacy givesThe narrow-model posterior mean is strongly pulled toward zero and its standard deviation is about $0.0995/\sqrt{n}$. The wide-model posterior mean is about $2.9703/\sqrt{n}$, very close to the observation, with standard deviation about $0.9950/\sqrt{n}$. Each model produces a markedly different posterior, so choosing between them requires their predictive evidence.

The requested conclusion is false for the printed parameter values. They give $V_0=1.01/n$ and $V_1=101/n$. Substituting $y=3/\sqrt{n}$ in the Bayes factor yieldsThus the Bayes factor favours $H_1$ by about $8.23$ to one, not $H_0$. Nor does the conclusion hold for arbitrary large $n_0,c$: both parameters enter the expression explicitly. With $n_0/n=100$, taking a much wider alternative, for example $c=1000$, would instead give $B_{01}\simeq 1.1563>1$. That is a different prior assumption and cannot repair the printed calculation silently.

With equal model prior probabilities, Bayes factor updating gives $w_0=B_{01}/(1+B_{01})$ and $w_1=1/(1+B_{01})$. The Bayesian model averaging posterior density is thereforeThis is a two-component mixture model of the normal distributions already derived. For the numerical observation above, $w_0\simeq 0.10829$ and $w_1\simeq 0.89171$. Consequently it is predominantly the wide-model posterior, not predominantly the narrow one.

Put ${\kappa }_i=n/(n+q_i)$. The posterior mean of the Gaussian practical-null mixture isNear zero, $B_{01}(0)>1$, so the narrow component has high posterior probability and ${\kappa }_0$ is small when $n_0\gg n$. This pulls the Bayesian model averaging posterior toward zero. For the given ratios, writing $z=\sqrt{n}y$ givesThus the order statement $y=O(n^{-1/2})$ alone does not guarantee strong pull toward zero: $z=3$ already has the opposite model preference.

As $|y|$ grows, $V_1>V_0$ makes $B_{01}(y)\to 0$, and the mixture model approaches $N({\kappa }_{1}y,1/(n+q_1))$. Its center is exactly ${\kappa }_{1}y$, and approximately $y$ only for a sufficiently diffuse wide prior, meaning $q_1\ll n$. Here ${\kappa }_1=100/101$, so the relative displacement is about one percent. Large observations alone do not remove this finite-prior shrinkage.

In a normal linear model, attach a continuous spike-and-slab prior to each regression coefficient. With suitably scaled predictors, introduce indicators ${\gamma }_j\sim \mathrm{Bernoulli}(\pi )$ and setThe narrow component describes practically negligible effects; the wide component permits substantial ones. Fit the joint Bayesian posterior of coefficients, indicators and any unknown residual variance. Bayesian model averaging gives shrinkage toward zero for poorly supported effects, while $\mathbb{P}({\gamma }_j=1\mid \mathcal{D})$ quantifies wide-component support. A shared Beta distribution prior on $\pi$ can represent uncertainty about how many effects are substantial.

Select on scientifically meaningful effect size, for example a high $\mathbb{P}(|{\beta }_j|>{\delta }_j\mid \mathcal{D})$ for a prechosen threshold in meaningful predictor units. Membership in the wide component alone does not imply a large realized effect: its normal distribution still permits values near zero. Correlated predictors also require interpretation of the joint Bayesian posterior, rather than treating each coefficient as an isolated test.