Past exam of the mathematics course of the University of Cambridge / 2009 / iii / Paper 36 / 2

Under the discrete uniform distribution, the observation has probability $1/N$ when its value is an available label, and zero otherwise. Therefore the likelihood function isThis is the discrete uniform endpoint posterior model's likelihood; all endpoints smaller than the observed label are excluded.

On its admissible support the likelihood function $1/N$ strictly decreases. Its maximum-likelihood estimate is consequently the smallest admissible endpoint:

A constant improper prior would give the formal posterior density kernel $N^{-1}{1}_{\{N\ge y\}}$. However,There is no normalized posterior distribution under this prior. The failure is not simply that the prior is improper: some improper priors give proper posteriors. Here posterior propriety itself fails after this single observation, so probabilities, quantiles and a posterior mean cannot be assigned to that kernel.

Write $H_k={\sum }_{j=1}^{k}1/j$, with $H_0=0$. The truncated uniform prior gives the posterior distributionIts posterior mean isEach term in the numerator of the expectation is $N/N=1$, which explains the exact count $M-y+1$.

Keep $y$ fixed. Integral bounds for the decreasing function $1/t$ give $H_M=\log M+O(1)$, so $H_M-H_{y-1}=\log M+O_y(1)$. ConsequentlyThe posterior mean thus grows like $M/\log M$ instead of approaching a data-determined limit. Even the posterior distribution mass below a fixed endpoint $n$ is $(H_n-H_{y-1})/(H_M-H_{y-1})\to 0$. Hence increasing the supposedly innocuous prior cutoff moves the inference to larger and larger endpoints. In an integral approximation its median is of order $\sqrt{yM}$, also depending strongly on the cutoff. This explains the sensitivity of the entire inference, not just of its posterior mean.

Multiplying the reciprocal improper prior by the likelihood function givesThus this discrete uniform endpoint posterior is proper. Its exact cumulative probability is $S_y^{-1}{\sum }_{k=y}^n{k}^{-2}$. Integral approximation gives $S_y\approx {\int }_y^{\infty }{t}^{-2}dt=1/y$ and ${\sum }_{k=y}^n{k}^{-2}\approx {\int }_y^n{t}^{-2}dt=1/y-1/n$, soThese approximations are useful when $y$ is moderately large and are not exact discrete identities. For example the exact untruncated median at $y=100$ is $199$. Although its quantiles are finite, its posterior mean is infinite, since its expectation numerator is the divergent sum ${\sum }_{N=y}^{\infty }1/N$.

The WinBUGS model uses a finite truncation of the reciprocal prior. The numbered lines have the following roles:

Combining that categorical distribution observation likelihood with the truncated reciprocal prior producesThe Markov chain Monte Carlo output approximates this posterior. The actual PDF separates the observation line from the preceding comment; it must remain an active stochastic statement.

The posterior distribution has a long right tail. Its posterior mean is pulled upward by relatively rare large endpoints and is sensitive to the artificial upper cutoff; its untruncated posterior mean does not exist. The median is a central quantile that stays finite as the cutoff is removed and is much less sensitive to extreme draws. It is also the Bayes estimator under absolute-error loss, whereas a mean corresponds to squared-error loss when the relevant expectation exists.

The median better represents the typical posterior endpoint in this strongly right-skewed distribution. It need not be preferable for every decision problem, since the loss function matters. For comparison, exact summation of the truncated kernel at cutoff $5000$ gives median $195$ and mean about $397.67$; the displayed simulation summaries are Monte Carlo estimates, not exact posterior calculations.