Past exam of the mathematics course of the University of Cambridge / 2009 / iii / Paper 36 / 1

The binomial likelihood is $p(x\mid \theta )=\left(\genfrac{}{}{0ex}{}{m}{x}\right){\theta }^x(1-\theta {)}^{m-x}$. Multiplication by the beta distribution prior density and Bayes' theorem give the posterior density kernel ${\theta }^{a+x-1}(1-\theta {)}^{b+m-x-1}$. Normalizing by the beta function yields Beta-binomial conjugacy:This is proper for $a,b>0$ and $0\le x\le m$, including counts at either boundary.

For the $m$ independent Bernoulli trials, the log likelihood function, apart from its constant, is $\ell (\theta )=x\log \theta +(m-x)\log (1-\theta )$. Its negative expected second derivative, the Fisher information, isFor $m>0$, the Jeffreys prior is proportional to its square root. Since $\mathrm{B}(1/2,1/2)=\pi$, its normalized form isThe Jeffreys prior is invariant under smooth one-to-one reparameterization as a measure. If $\varphi =h(\theta )$, then $I_{\varphi }(\varphi )=I_{\theta }(\theta )(d\theta /d\varphi {)}^2$, so $\sqrt{I_{\varphi }}d\varphi =\sqrt{I_{\theta }}|d\theta |$. Thus recomputing it in the new coordinate gives precisely the transformed prior, rather than a new prior. In several dimensions the corresponding density is $\sqrt{\det I(\theta )}$, with the same Jacobian transformation property.

Put $\alpha =a+x$, $\beta =b+m-x$, and $S=\alpha +\beta =m+a+b$. Integrate the future binomial distribution against the current beta distribution posterior density:This is the beta-binomial distribution. For positive integer $a,b$, the gamma function values reduce to factorials, givingUsing $(S-1)!=(S-1)(S-2)!$ and $(S+n-1)!=(S+n-1)(S+n-2)!$ rewrites this asSubstitution of $\alpha ,\beta ,S$ gives exactly the two required factors. The probability is zero outside $0\le y\le n$.

With a uniform beta distribution prior, the posterior predictive distribution for another batch of the same size isBut $\mathrm{B}(x+1,m-x+1)=1/[(m+1)(\genfrac{}{}{0ex}{}{m}{x})]$. HenceThe displayed expression is symmetric in $x,y$. This is the uniform-count predictive property: the joint batch counts are exchangeable and both marginal counts are uniform. Conditional reversal is not a general property of arbitrary priors; it holds here because those marginal probabilities coincide.

Before collecting any observations, a uniform success-chance prior distribution givesThus the uniform-count predictive property assigns equal probability to every possible number of successes in a fixed-size batch under the prior predictive distribution. It is uniform over counts, not over all ordered binary sequences. After observing $x$ successes in $m$ trials, its familiar one-step posterior predictive probability is $(x+1)/(m+2)$; the count-uniform property refers to prediction before those data are observed.

For the uniform prior, write $s=x+y$ for the pooled success count in $m+n$ trials. Conditional on $s$, exchangeability makes all placements of the successes equally likely. The successes in the second batch therefore have a hypergeometric distribution, and the factor in question isThis is hypergeometric allocation of exchangeable Bernoulli counts: sample $n$ positions without replacement from a population of $m+n$ positions containing $s$ successes. The uniform-count predictive property gives $\mathrm{Pr}(X+Y=s)=1/(m+n+1)$ and $\mathrm{Pr}(X=x)=1/(m+1)$, soAs $y$ varies with $x$ fixed, $s=x+y$ also varies. Therefore $B$ alone is not a normalized hypergeometric distribution over $y$; its normalization factor is essential.