Past exam of the mathematics course of the University of Cambridge / 2008 / iii / Paper 44 / 1

Let the prior odds be $\mathbb{P}(T)/\mathbb{P}(F)=R$, so $\mathbb{P}(T)=R/(1+R)$. The Type I error gives $\mathbb{P}(P\mid F)=\alpha$, while the statistical power gives $\mathbb{P}(P\mid T)=1-\beta$. By Bayes' theorem,This is a posterior probability of the null, not the false-positive probability $\alpha$ conditional on the null. The distinction is the base-rate effect in a positive study.

The positive result's posterior odds of true to false areThus being more likely true requires $\boxed{R>\alpha /(1-\beta )}$. The question's weak inequality is necessary; equality gives a tie rather than a strict preference.

Under the null, $Y$ has the standard normal distribution, so the Type I error is $\alpha ={\mathbb{P}}_0(Y>1.65)=1-\Phi (1.65)\simeq 0.05$. Under the alternative, the Type II error is $\beta ={\mathbb{P}}_1(Y\le 1.65)=\Phi (1.65-2.49)=\Phi (-0.84)\simeq 0.20$. Hence $\boxed{\alpha \simeq 0.05,\beta \simeq 0.20}$, using the supplied rounded normal probabilities.

For these two simple hypotheses the Bayes factor is the ratio of their normal probability densities:Therefore $\boxed{B_{10}(y)=e^{2.49(y-1.245)}}$, and the posterior odds are $R{B}_{10}(y)$.

Yes: replacing the observed statistic by a significance label loses information. The binary positive event has Bayes factor $(1-\beta )/\alpha \simeq 16$, but the observed-statistic Bayes factor varies continuously with $y$. A result just at the threshold has $B_{10}(1.65)\simeq 2.74$, while $B_{10}(3)\simeq 79.0$. Both would receive the same positive label.

The binary calculation is correct for that deliberately coarsened observation; its problem is treating it as the whole evidence when the statistic is available. This is the point developed in Goodman and Greenland's response.

Conditional on a false relationship, the independent studies are each negative with probability $1-\alpha$. The complement of at least one positive is that all are negative. Hence the familywise error rate is $\boxed{1-(1-\alpha {)}^n}$.

Let $A_n$ denote the event that at least one study is positive. Conditional independence gives $\mathbb{P}(A_n\mid T)=1-{\beta }^n$ and $\mathbb{P}(A_n\mid F)=1-(1-\alpha {)}^n$. Updating the prior odds by this event's Bayes factor givesThese are the odds under at-least-one-positive selection, not the odds using every study's actual result.

For $0<\alpha ,\beta <1$, both event probabilities tend to one, so the posterior odds tend to $\boxed{R}$. If $1-\beta >\alpha$, the event's Bayes factor decreases toward one. To see monotonicity, put $a=1-\alpha >\beta$ and writeThe last quotient is a weighted average of the decreasing values $(\beta /a{)}^k$, with weights $a^k$. Adding the next, smaller value decreases the average.

Thus the selection rule becomes less informative as more teams try: eventually a positive study is almost certain even under the null. This explains the hotter-field interpretation in Ioannidis's model. It describes evidence retained by this particular selection event, rather than proving that every individual study becomes less reliable or that the full accumulated evidence gets weaker.

Consistent independent replications should strengthen confidence; contradictory results can weaken it. The change depends on the actual evidence. If $k$ of $n$ studies are positive and all labels are available, the posterior odds areFor the normal model, retaining the full independent statistics instead givesEach study multiplies the existing odds by its own Bayes factor. Reading a complete synthesis is therefore different from learning only that somebody obtained a positive result. Publication bias, dependence between studies and changes in study design must be accounted for before applying the independent-product formula.