Past exam of the statistics course of the University of Oxford / 2023 / 3

Given the observed data, a $100(1-\alpha )\%$ credible interval $[a,b]$ for the scalar parameter satisfiesThe probability refers to the posterior distribution, proportional to $\pi (\theta ){\prod }_{i}f(x_i\mid \theta )$ on the parameter space. It conditions on the observations; it is not the repeated-sampling coverage probability definition of a confidence interval.

An equal-tailed credible interval for a continuous posterior distribution has probability $\alpha /2$ below its lower endpoint and $\alpha /2$ above its upper endpoint. If $F$ is the posterior cumulative distribution function, the endpoints are $F^{-1}(\alpha /2)$ and $F^{-1}(1-\alpha /2)$.

A highest posterior density interval is a credible interval consisting of points of highest posterior density: every included point has density at least as high as every excluded point, apart from null-set and boundary choices. It is a posterior highest density region when that region is an interval. For a continuous unimodal density with two interior endpoints, their densities agree. A mode at a support boundary can instead force one endpoint to that boundary; a multimodal posterior can have a disconnected highest-density region rather than one interval. Equal tails and highest density need not give the same interval.

Writing $s={\sum }_i{x}_i$, the Poisson distribution likelihood is proportional to ${\theta }^s{e}^{-n\theta }$. Multiplying by the gamma distribution prior givesThis is Poisson-gamma conjugacy, with the rate convention made explicit:Its normalized density is $(\lambda +n{)}^{r+s}{\theta }^{r+s-1}{e}^{-(\lambda +n)\theta }/\Gamma (r+s)$ for $\theta >0$.

The posterior mean is a convex combination of the prior mean and the usual Poisson maximum-likelihood estimate:Both weights are positive and sum to one, so the mean lies between $r/\lambda$ and $\bar{x}$, strictly between them unless they coincide. If all counts are zero, $\bar{x}=0$ is the boundary estimate in the usual nonnegative Poisson parameter space; with a strictly positive parameter restriction it represents the likelihood supremum at that boundary. The displayed posterior-mean relation is valid in either interpretation.

Let $M=\max \{b,y_1,\dots ,y_n\}$. The continuous uniform distribution likelihood contributes ${\varphi }^{-n}{1}_{\{\varphi \ge {\max }_i{y}_i\}}$, while the Pareto distribution prior contributes ${\varphi }^{-a-1}{1}_{\{\varphi \ge b\}}$. Their product proves Uniform-Pareto conjugacy:Thus the posterior distribution is Pareto with shape $a+n$ and lower bound $M$. Its density is strictly decreasing on its support, so the highest posterior density interval begins at $M$. Since the upper-tail probability is $(M/u{)}^{a+n}$, the required upper endpoint satisfies $(M/u{)}^{a+n}=0.05$. The Pareto highest density interval is therefore

Write $m=r+s$ and $\beta =\lambda +n$. Conditional independence of the next count given $\theta$ allows us to average its Poisson distribution over the posterior distribution:This Poisson-gamma mixture is the posterior predictive distribution. In the stated negative binomial distribution convention its parameters areIts expected value $m/\beta$ equals the posterior mean of $\theta$, while its variance $m/\beta +m/{\beta }^2$ includes both conditional Poisson variation and uncertainty in $\theta$.