Past exam of the statistics course of the University of Oxford / 2014 / 3

Given observed data $x$, a credible interval $[L(x),U(x)]$ has posterior probability $1-\alpha$:For a continuous posterior distribution with probability density function $\pi (\theta \mid x)$, this is ${\int }_{L(x)}^{U(x)}\pi (\theta \mid x)d\theta =1-\alpha$. The probability is conditional on the observed data and concerns the statistical parameter under its posterior distribution; it does not assert frequentist coverage probability.

An equal-tailed credible interval allocates equal posterior probability to its two excluded tails:For a continuous strictly increasing posterior cumulative distribution function $F$, the endpoints are $F^{-1}(\alpha /2)$ and $F^{-1}(1-\alpha /2)$.

A highest posterior density interval instead includes the parameter values of greatest posterior density: there is a cutoff $c$ such that the retained region is a superlevel set of $\pi (\theta \mid x)$, of posterior mass $1-\alpha$. Every interior retained value has density at least as large as every excluded value, apart from possible ties. For a unimodal continuous density this is an interval; when both endpoints lie inside the support they have equal posterior density. For a multimodal posterior distribution, a highest-density region may be a union of intervals, rather than one interval. Equal tail probabilities and highest density are different criteria.

Let $s={\sum }_i{x}_i$, $A=a+s$ and $B=b+n$. Conditional independence gives the Poisson distribution's likelihoodMultiply by the prior gamma distribution's density and discard factors independent of $\theta$:Normalizing with the gamma function identifiesHere $A=a+s$ is the shape and $B=b+n$ is the rate, not the scale. This is conjugate prior updating for a Poisson distribution.

The mean of a shape-rate gamma distribution is shape divided by rate. Hence the posterior mean isBoth weights are positive and sum to one. This is a convex combination of the prior mean and the sample mean, provingFor $n\ge 1$ it lies strictly between them if the two means differ, and equals both if they agree. The rate $b$ acts as the prior weight relative to the $n$ observed exposure units.

Let $F$ be the cumulative distribution function of a gamma distribution with shape $A=a+{\sum }_i{x}_i$ and rate $B=b+n$; explicitly,The posterior distribution is continuous and has positive density on $(0,\infty )$, so its quantiles give the requested equal-tailed credible interval:Equivalently, $2B\theta \mid x\sim {\chi }_{2A}^2$, so the endpoints can be written as the corresponding chi-squared distribution quantiles divided by $2B$. Either form excludes posterior probability $\alpha /2$ from each tail.

No. A permitted counterexample is $a=1$ with all observed counts zero. Then $A=1$ and the posterior distribution is an exponential distribution with rate $B=b+n$, whose density decreases strictly from the left boundary. Its equal-tailed credible interval isHere $L>0$. This excludes values just above zero whose posterior density is larger than that of retained values near $U$, violating the highest-density criterion. The actual highest posterior density interval starts at the support boundary and solves $1-e^{-Bh}=1-\alpha$:Writing zero as the endpoint causes no problem because a single point has zero posterior probability. Thus even within the gamma family the two intervals need not agree. More generally, for an interior unimodal gamma distribution, highest-density endpoints have equal densities, whereas equal-tailed endpoints need not.

Use $A=a+{\sum }_i{x}_i$ and $B=b+n$ as above. The future observation is independent of the old observations conditional on $\theta$, so the posterior predictive probability is obtained by integrating its Poisson distribution against the posterior distribution of $\theta$:The last integral is the normalizing gamma function integral, equal to $\Gamma (A+k)/(B+1{)}^{A+k}$. This is a negative binomial distribution in the count-of-failures convention with shape $A$ and success probability $B/(B+1)$; the gamma-function expression remains valid for noninteger $A>0$. As checks, its mean is $A/B$, and the law of total variance gives variance $A/B+A/B^2$. The additional term is the contribution of posterior uncertainty in $\theta$.