Past exam of the statistics course of the University of Oxford / 2021 / 3 / a

Set $S={\sum }_i{x}_i$ and $N=nm$. Conditional independence of the binomial distribution observations gives the likelihood functionThe prior distribution is the beta distribution with parameters $(1,\beta )$. By Bayes' theorem, multiplying the prior density by the likelihood function gives a posterior density proportional toRecognizing and normalizing this beta distribution givesfor $0<\theta <1$. This is Beta-binomial conjugacy; the original binomial coefficients cancel in the normalization because they are independent of $\theta$.

The prior mean, posterior mean, and maximum-likelihood estimate areFor $0<S<N$, the last follows by maximizing $S\log \theta +(N-S)\log (1-\theta )$. At $S=0$ or $S=N$, use the usual binomial likelihood convention that admits the endpoints $0$ and $1$ as maximum-likelihood estimates; if the parameter domain is strictly $(0,1)$, these are boundary suprema rather than attained maxima. The posterior mean remains interior in all cases.

TakinggivesSince $N=nm>0$ and $\beta +1>0$, $0<\lambda <1$. Thus the posterior mean is a convex combination of the prior mean and observed binomial proportion, with larger prior weight when $\beta +1$ is large relative to $N$.

The prior mean $1/(\beta +1)$ decreases as $\beta$ increases. More strongly, its cumulative distribution function and upper-tail probability areFor every fixed $t$, that upper-tail probability decreases with $\beta$. The statistician increasingly favors smaller values of $\theta$; as $\beta \to \infty$, the prior distribution concentrates near zero. For $\beta >1$, its variance $\beta /[(\beta +1{)}^2(\beta +2)]$ also decreases. This is concentration of a beta prior near zero.

With the data fixed, the derivatives areThus the prior weight increases and the posterior mean decreases. Their limits are $\lambda \to 1$ and ${\tilde{\theta }}_1\to 0$. Both the changing prior mean and the increasing weight contribute to the change in the posterior mean.