= Gaussian-process marginal likelihood
{c}
Let latent values $f$ have a <multivariate normal distribution> $N(m,K)$, and observe $y=f+\varepsilon$ with <independent> <Gaussian noise> $\varepsilon\sim N(0,E)$. Integrating over $f$ gives $y\sim N(m,C)$, where $C=K+E$. If $C$ is <positive-definite>, its log-<marginal likelihood> is
$$
\log p(y)=-\frac12(y-m)^TC^{-1}(y-m)-\frac12\log\det C-\frac n2\log(2\pi).
$$
The first term measures the residual under the proposed <covariance matrix>; the <determinant> term accounts for the <volume> of the proposed observation law. Both matter when comparing <covariance function> parameters. With a single observation, $m=0$ and $C=v>0$, this reduces to $-y^2/(2v)-\log(v)/2-\log(2\pi)/2$.<Gaussian Processes for Machine Learning by Carl Edward Rasmussen and Christopher K. I. Williams>[Chapter 2, equation (2.30); nonzero-mean extension by centering]{ref}
A <positive-definite> observation-noise <covariance matrix> $E$ makes $C$ <positive-definite> even when $K$ is singular; mutually independent errors with strictly positive <variances> are one sufficient case. If $C$ is singular, a Gaussian observation law still exists but has no ordinary density on the full observation space; the inverse-and-determinant formula above is then not valid. This is a likelihood for kernel and <mean> parameters after latent-value integration, rather than an assertion that estimated parameters have no further uncertainty.
Back to article page
