Past exam of the options I course of the University of Oxford / 2013 / O / 2

Let $s={\sum }_{j=1}^n{x}_j$. The Bernoulli distribution likelihood is proportional to ${\theta }^s(1-\theta {)}^{n-s}$. Multiplication by the beta distribution prior givesNormalizing with the beta function therefore givesThe parameters remain positive. This is Beta-binomial conjugacy: successes add to the first shape parameter and failures to the second.

A $100(1-\alpha )\%$ credible set is a measurable subset $C(x)\subseteq (0,1)$ with posterior probability $1-\alpha$:In ordinary language, given the observed data and the chosen prior model, the probability that the parameter lies in this set is $1-\alpha$. Such a set need not be an interval or be unique; its defining requirement is posterior mass.

Let $F_{A,B}$ denote the distribution function of the posterior beta distribution, with $A=a+s$ and $B=b+n-s$. This distribution is continuous with positive density on $(0,1)$. Its equal-tailed credible interval isCompute the two quantiles by solving ${\int }_0^{{\theta }_L}\pi (\theta \mid x)d\theta =\alpha /2$ and ${\int }_{{\theta }_U}^1\pi (\theta \mid x)d\theta =\alpha /2$. Each excluded tail has mass $\alpha /2$, leaving posterior mass $1-\alpha$ between the endpoints.

The further observation is independent of the earlier observations conditional on $\theta$. Integrate its conditional success probability against the posterior:The given factorial formula for the beta function gives this ratio as $A/(A+B)$. Thus the posterior predictive probability isIt is the posterior mean of the Bernoulli success parameter. Independence here is conditional: sharing the uncertain parameter generally makes past and future observations dependent before conditioning on it.