Past exam of the mathematics course of the University of Cambridge / 2013 / iii / Paper 34 / 1

Each observation has uniform distribution density ${\theta }^{-1}{1}_{\{0<y_j<\theta \}}$. Independence makes the likelihood function their product. For positive observations the support reduces to $\theta >M$, givingFor data outside the positive orthant the likelihood is zero. Changing the convention at the endpoint does not change a continuous posterior distribution.

Multiplying the likelihood function by the Pareto distribution prior givesThe integral of this kernel is $m^{-(\alpha +n)}/(\alpha +n)$, so the normalized posterior distribution isThus it is $\mathrm{Pareto}(\alpha +n,m)$. This proves uniform-Pareto conjugacy: applying Bayes theorem preserves the family of prior distributions.

The joint prior predictive distribution is the Bayesian model evidence, obtained by integrating over the shared parameter:It is zero otherwise. This uniform-Pareto model evidence is a joint density, not the product of separately marginalized observation densities: mixing over the common parameter induces dependence.

Exchangeability means that the joint prior distribution is invariant under breed relabelling:for every permutation $\pi$. The labels carry no prior information about maximum size. This is reasonable for comparable breeds before observing their measurements when no covariates or biological knowledge distinguish them.

Exchangeability does not imply independence. In a hierarchical Bayesian model, the parameters can be independent conditional on a shared hyperparameter but dependent after it is integrated out. Known systematic biological differences would call for a model of those differences, with exchangeability only for the remaining unexplained variation.

Use the intended hierarchical Bayesian model: the breed parameters are conditionally independent given $\alpha ,\beta$, and observations are independent given their breed parameters. Put $m_i=\max (\beta ,M_i)$. The uniform-Pareto model evidence factorizes over breeds:Identical marginal prior distributions alone would not determine this product; conditional independence is the additional assumption.

The Type II maximum likelihood estimator maximizes the Bayesian model evidence after integrating out the breed parameters:For positive sample sizes define $S={\sum }_i\log (m_i/\beta )\ge 0$. Up to an additive constant, the log-likelihood isDifferentiation gives the interior score equationAlso ${\ell }^{\prime \prime }(\alpha )=-I/{\alpha }^2+{\sum }_i(\alpha +n_i{)}^{-2}<0$. If $S>0$, the derivative decreases from infinity to $-S$, so a unique finite maximum exists. When all sample sizes equal $n$, the equation becomes $In/[\alpha (\alpha +n)]=S$, givingFor $S=0$ no finite maximum exists. Plugging this hyperparameter estimate into the breed posterior distributions is an Empirical Bayes method.

Here $m_i=\beta$ for every breed, so $S=0$ andThe Bayesian model evidence increases towards its supremum as $\alpha \to \infty$. ThusFor a Pareto distribution, $\mathbb{P}(\theta >\beta (1+\epsilon ))=(1+\epsilon {)}^{-\alpha }\to 0$. The limiting empirical prior, and each corresponding posterior distribution, collapses onto $\beta$.

The boundary result is coherent within the assumed model: the observations favour the smallest allowed upper limits. Nevertheless, reporting exact concentration on a prespecified bound is overconfident for finite data and ignores hyperparameter uncertainty. A proper hyperprior on $\alpha$, sensitivity analysis for $\beta$, or a scientifically justified restriction on concentration avoids treating this limit as certain biological knowledge.

Fit two hierarchical Bayesian models to the same observations, one with the bounded uniform distribution sampling density and the other with an exponential distribution density. Specify rate versus mean in the exponential model and assign appropriate, scientifically comparable proper prior distributions; that parameter is no longer a literal maximum size. Obtain posterior distributions, for example by Markov chain Monte Carlo, and check convergence.

At the same observational level in both models use the Bayesian deviance $D(\varphi )=-2\log p(\mathbf{y}\mid \varphi )$, excluding prior densities. Retain the same likelihood constants and consistently either condition on breed effects or integrate them out. Estimate $\overline{D}=\mathbb{E}[D(\varphi )\mid \mathbf{y}]$, evaluate $D(\overline{\varphi })$ at the posterior mean, and compute the effective parameter count in DIC:Smaller DIC favours the fit–complexity tradeoff. Supplement it with posterior predictive checks; the uniform model's parameter-dependent support and the hierarchical structure make the criterion a diagnostic rather than an automatic definitive decision.