Past exam of the mathematics course of the University of Cambridge / 2010 / iii / Paper 36 / 1

If ${\pi }_0=\mathbb{P}(H_0)$ and ${\pi }_1=\mathbb{P}(H_1)=1-{\pi }_0$, the prior odds on the null hypothesis areThese odds concern which model is true before seeing the observations. They are separate from the prior distribution of the success probability within the alternative model.

Use the orientation that favours the null hypothesis:For a composite model the denominator is its Bayesian model evidence, obtained by integrating the likelihood against the model's prior distribution. This Bayes factor exceeds one when the observed data have greater marginal probability under $H_0$. By Bayes' theorem,Thus the posterior odds equal the prior odds multiplied by the Bayes factor. The reciprocal orientation instead gives $B_{10}=1/B_{01}$.

Conditional on the success probability, the count has binomial distribution. Integrating its likelihood under the uniform prior distribution, or equivalently a beta distribution with parameters $(1,1)$, givesThe prior predictive distribution is therefore uniform over the possible counts. Excluding the single point $\theta =1/2$ does not alter this integral because a continuous prior distribution gives that point zero probability. This calculation is the marginalization underlying Beta-binomial conjugacy.

Under the null hypothesis, $Y\sim \mathrm{Bin}(n,1/2)$, with expected value $n/2$ and variance $n/4$. Approximating its probability at an integer by the normal distribution density over a unit-width cell givesDividing by the prior predictive distribution from the preceding part gives the uniform-alternative binomial Bayes factorThe second approximation replaces $n+1$ by $n$. Both are large-sample approximations for counts in the central range, rather than exact identities at extreme counts.

The normal distribution standardized statistic for the binomial distribution count isBecause the alternative is two-sided, the approximate p-value is $2\{1-\Phi (3)\}\approx 0.0027$. A continuity correction replaces $3$ by $(149.5)/50=2.99$ and gives approximately $0.0028$, with the same conclusion. Thus the result is statistically significant at both the 5% and 1% levels. The p-value is a null tail probability, not a posterior probability that the null hypothesis is true.

Substituting the standardized deviation into the Bayes factor approximation givesThis is only a slight preference for the alternative, despite the small p-value. The two procedures measure different things: the p-value sums a tail under the null hypothesis, while the Bayes factor compares probabilities of the observed count under both models. The uniform alternative prior distribution spreads its probability over many success probabilities that fit the observations poorly. Only a narrow range around $0.515$ has large likelihood, and integrating over the whole alternative penalizes this diffuse prediction.

The prefactor makes the distinction especially clear. At a fixed standardized deviation $z$, the p-value stays approximately $2\{1-\Phi (|z|)\}$, whereas $B_{01}\approx \sqrt{2n/\pi }{e}^{-z^2/2}$ eventually grows with $n$. A fixed diffuse alternative can consequently favour the point null even while a conventional test rejects it. This is the same prior-width mechanism illustrated by the Jeffreys-Lindley paradox for a Gaussian point null.

An alternative that allows small departures from chance, while assigning little probability to near-perfect prediction or near-perfect failure, is more plausible than a uniform prior distribution. For example, a symmetric beta distribution $\mathrm{Beta}(a,a)$ with $a>1$ is centered at $1/2$, withChoosing $a=50$ gives a prior standard deviation of about $0.05$ and concentrates on modest deviations. Its concentration should represent substantive beliefs specified before examining this experiment, rather than being tuned to the observed count. A continuous prior distribution centered at $1/2$ still places no point mass at $H_0$; the separate model prior probability supplies that mass.

No. A Bayes factor favouring the alternative means $B_{01}<1$, but the posterior odds are $B_{01}{O}_{01}$. They are less than one only if $B_{01}<1/O_{01}$. For example, prior odds of $1000:1$ on the null hypothesis, combined with a Bayes factor of $100:1$ in favour of the alternative, still give posterior odds of $10:1$ on the null hypothesis. An unusual claim can require substantial evidence to overcome low initial plausibility.