Past exam of the advanced topics in statistical machine learning course of the University of Oxford / 2025 / 3 / c / i / Solution

Use only the first $n-1$ observations to defineThus the full noisy covariance matrix is $C_n=\left(\begin{array}{cc}A& b\\ b^T& c\end{array}\right)$ and the full cross-covariance vector is $(a^T,d{)}^T$. SetThe scalar $s$ is the Schur complement of $A$. Substituting the block inverse into the Gaussian process regression posterior variance givesConsequently,andFor interpretation, after the first $n-1$ observations the remaining covariance between $f_{\ast }$ and $f(x_n)$ is $r=d-b^T{A}^{-1}a$, while the conditional variance of the new noisy observation is $s$. Thus the reduction is the squared remaining covariance divided by the remaining observation variance.