Past exam of the mathematics course of the University of Cambridge / 2023 / iii / Paper 216 / 1

The likelihood function and prior distribution give the posterior densityCompleting the square in the quadratic form givesThus Gaussian conjugacy for a normal linear model yields

After integrating out the multivariate normal distribution $\beta$, the marginal distribution isUp to terms independent of ${\sigma }^2$, the log-likelihood isDifferentiating and setting the result to zero gives the maximum marginal likelihood estimatorThis is an Empirical Bayes method because the estimated hyperparameter is then inserted into the prior and posterior distributions.

For a decision $x$ in the unit ball, conditional expectation under the posterior givesThe symmetric positive semidefinite matrix $C+m{m}^T$ has a Rayleigh quotient maximized over $\Vert x\Vert \le 1$ by any unit eigenvector corresponding to its largest eigenvalue. Bayes decision rule therefore chooses any such eigenvector for the stated utility.

For the first $i$ observations, writeBy Gaussian conjugacy for a normal linear model, the prefix posterior isCompute a Cholesky decomposition of $A_0={\Sigma }^{-1}$ once. If $x_i^T$ is row $i$ of the design matrix, thenThe Rank-one Cholesky update obtains a triangular factor $A_i=L_i{L}_i^T$ from $L_{i-1}$ in $O(p^2)$ operations. Two triangular solves give $m_i=A_i^{-1}{b}_i$, and, for $z_i\sim N(0,I_p)$, another solve givesThe initial factorization costs $O(p^3)$ and all $n$ updates and samples cost $O(n{p}^2)$. This is within the requested $O(p^3+n{p}^2+n^{2}p)$ bound.

Put $z_i=\log Y_i$. The Weighted graph Laplacian $L_J$ of the tree is defined byMultiplication of the Gaussian likelihood by the prior shows that, conditionally on the precision parameter $\tau$,Completing the square therefore givesAs a function of $\tau$, the posterior density isso, in shape-rate notation,The precision matrix $M$ has the sparsity pattern of a tree. A sparse Cholesky decomposition and its triangular solves have $O(n)$ cost and storage on this graph, while the gamma update also costs $O(n)$. Hence each systematic-scan Gibbs sampler iteration costs $O(n)$.