= Solution
{c}
Write $a=(2,1)^T$, so the data map is $a^Tu$. The <Gaussian likelihood> and <prior distribution> give
$$
\pi^m(u)\propto\exp\left[-\frac12u^Tu-\frac{(m-a^Tu)^2}{2\delta^2}\right].
$$
<Completing the square>, or using <Gaussian conjugacy for a normal linear model>, identifies a <multivariate normal distribution> with <precision matrix> $C_\delta^{-1}=I_2+\delta^{-2}aa^T$. Since $a^Ta=5$, its <posterior mean> and <covariance matrix> are
$$
\boxed{u\mid m\sim\mathcal N(b_\delta,C_\delta).}
$$
The parameters are
$$
\begin{aligned}
b_\delta&=\frac{m}{5+\delta^2}\begin{pmatrix}2\\1\end{pmatrix},\\
C_\delta&=I_2-\frac{aa^T}{5+\delta^2}
=\frac1{5+\delta^2}\begin{pmatrix}1+\delta^2&-2\\-2&4+\delta^2\end{pmatrix}.
\end{aligned}
$$
For example, $(I_2+\delta^{-2}aa^T)C_\delta=I_2$, which checks the square completion.
The <orthogonal vectors>
$$
e_\parallel=\frac1{\sqrt5}(2,1)^T,\qquad
e_\perp=\frac1{\sqrt5}(-1,2)^T
$$
are <eigenvectors> of the <covariance matrix>. Their respective <variances> are
$$
\boxed{\operatorname{Var}(e_\parallel^Tu\mid m)=\frac{\delta^2}{5+\delta^2},\qquad
\operatorname{Var}(e_\perp^Tu\mid m)=1.}
$$
They are independent under the <posterior distribution>, by <independence of uncorrelated jointly normal variables>. Thus uncertainty is different in the two directions even for positive noise: the data constrain $e_\parallel$, whereas $e_\perp$ spans the <null space> of the data map.
As $\delta\to0$, the <posterior distribution> converges to the law of
$$
\frac m5(2,1)^T+Ze_\perp,\qquad Z\sim\mathcal N(0,1).
$$
This is supported on $2u_1+u_2=m$, and its <posterior mean> is the <minimum-norm least-squares solution>. The <prior distribution> still determines the distribution along the unobserved <null space>; it has not disappeared. This example of a <Gaussian posterior in the zero-noise limit> distinguishes exact recovery of an observed component from recovery of the whole unknown.
Back to article page
