Past exam of the mathematics course of the University of Cambridge / 2019 / iii / Paper 219 / 4

For one object, the probabilistic graphical model factorization isEach factor is the normal distribution density specified by the model. This factorization displays the conditional independences of the latent variables ${\xi }_i,{\eta }_i$ and the noisy observations $x_i,y_i$.

With the stated flat priors, the full joint density, up to a constant, isThe priors on $\alpha ,\beta ,$ and $\mu$ contribute constants on $\mathbb{R}$, while those on the two variances contribute constants on $(0,\infty )$. These are improper priors, so posterior propriety must be checked; the full-rank, sufficiently large-data case used below is proper.

For each $i$, place ${\xi }_i$ and ${\eta }_i$ inside a plate replicated $N$ times. The directed edges are represented byThe shaded observed nodes are $x_i,y_i$; the unshaded nodes ${\xi }_i,{\eta }_i$ are latent; and $\alpha ,\beta ,{\sigma }^2,\mu ,{\tau }^2$ lie outside the plate. This is the probabilistic graphical model encoded by the joint factorization.

Every move can be drawn from a full conditional distribution, producing a Gibbs sampler with acceptance probability one. Write $s_{x,i}^2={\sigma }_{x,i}^2$ and $s_{y,i}^2={\sigma }_{y,i}^2$. First update independentlyand thenLet $X$ have rows $(1,{\xi }_i)$ and $\eta =({\eta }_i)$. Update the linear regression coefficients jointly byand updateFinally, the flat positive variance priors give the following full conditionals, each an inverse-gamma distribution:A systematic sweep in the displayed order, using the newly sampled values immediately, defines the chain. The shapes are positive for $N>2$; full column rank of $X$ is also required.