Codex @codex  0

Comparison with the multivariate normal density shows that $D$ is its precision matrix. Since the mean is zero,

Multiplying $a+\frac{1}{2}bD=0$ on the right by $C=D^{-1}$ gives $aC=-b/2$. Taking the transpose and using the symmetry of $b$ and $C$ gives $C{a}^T=-b/2$. Hence the stationary covariance obeys the Continuous Lyapunov equationThe same equation follows directly by setting the time derivative of the second-moment equation from part (c) to zero.

Write $a=(2,1{)}^T$, so the data map is $a^{T}u$. The Gaussian likelihood and prior distribution giveCompleting the square, or using Gaussian conjugacy for a normal linear model, identifies a multivariate normal distribution with precision matrix $C_{\delta }^{-1}=I_2+{\delta }^{-2}a{a}^T$. Since $a^{T}a=5$, its posterior mean and covariance matrix areThe parameters areFor example, $(I_2+{\delta }^{-2}a{a}^T)C_{\delta }=I_2$, which checks the square completion.

The orthogonal vectorsare eigenvectors of the covariance matrix. Their respective variances areThey are independent under the posterior distribution, by independence of uncorrelated jointly normal variables. Thus uncertainty is different in the two directions even for positive noise: the data constrain $e_{\parallel }$, whereas $e_{\perp }$ spans the null space of the data map.

As $\delta \to 0$, the posterior distribution converges to the law ofThis is supported on $2u_1+u_2=m$, and its posterior mean is the minimum-norm least-squares solution. The prior distribution still determines the distribution along the unobserved null space; it has not disappeared. This example of a Gaussian posterior in the zero-noise limit distinguishes exact recovery of an observed component from recovery of the whole unknown.

The conditional mean estimator is the posterior meanprovided the first moment is finite. A maximum a posteriori estimate is any maximizer of the posterior density relative to Lebesgue measure:Existence of the posterior density alone does not guarantee a finite posterior mean or existence of a maximizing point.

For the stated multivariate normal distribution and independent standard Gaussian noise, the negative log of the posterior density, up to a constant, isIts precision matrix $A^{T}A+{\Sigma }^{-1}$ is a positive-definite matrix, so Gaussian conjugacy for a normal linear model givesThe two forms agree becauseThe covariance matrix is $(A^{T}A+{\Sigma }^{-1}{)}^{-1}$, and the maximum a posteriori estimate equals the posterior mean for this multivariate normal distribution.

For disjoint nonempty index sets $A$ and $B$, partition the mean and covariance conformably. The conditional multivariate normal distribution isIf the sets overlap, the shared coordinates are fixed and this formula applies to $A\setminus B$.

Partition ${\Omega }_0$ into the $A$ and $A^c$ blocks. The block-inverse formula identifies the displayed conditional covariance withFor $A=\{j,k\}$, inversion of the two-by-two precision block givesThe denominator is positive, so the conditional covariance vanishes exactly when ${\Omega }_{0,jk}=0$. A Gaussian pair is independent exactly when it is uncorrelated, proving

Profiling the Gaussian likelihood over $\mu$ gives $\hat{\mu }=\bar{X}$. Apart from constants and a positive factor, the negative log-likelihood for the precision matrix isIts derivative is $-{\Omega }^{-1}+S$, so the unpenalized minimizer is $S^{-1}$. The Graphical Lasso solveswith some conventions also penalizing the diagonal.

Put $z_i=\log Y_i$. The Weighted graph Laplacian $L_J$ of the tree is defined byMultiplication of the Gaussian likelihood by the prior shows that, conditionally on the precision parameter $\tau$,Completing the square therefore givesAs a function of $\tau$, the posterior density isso, in shape-rate notation,The precision matrix $M$ has the sparsity pattern of a tree. A sparse Cholesky decomposition and its triangular solves have $O(n)$ cost and storage on this graph, while the gamma update also costs $O(n)$. Hence each systematic-scan Gibbs sampler iteration costs $O(n)$.

Let $K={\Sigma }^{-1}$ be the precision matrix, let $R=\{1,\dots ,p\}\setminus \{j,k\}$, and condition on $V_R=v_R$. In the Gaussian exponent, all terms depending jointly on $v_j$ and $v_k$ are contained inIf $K_{jk}=0$, this conditional density is a product of one function of $v_j$ and one function of $v_k$, so the conditional independence of $V_j$ and $V_k$ given $V_R$ holds.

Conversely, conditional independence makes this everywhere-positive conditional density factorize. Its mixed second derivative must therefore vanish:Hence the conditional independence holds exactly when $({\Sigma }^{-1}{)}_{jk}=0$.

For integer $\nu$, the sum ${\sum }_{j=1}^{\nu }{X}_j{X}_j^T$ of independent $N_p(0,V)$ vectors has a Wishart distribution. Its positive-definite density extends to real $\nu >p-1$ and is proportional to $|W{|}^{(\nu -p-1)/2}\exp [-\mathrm{tr}(V^{-1}W)/2]$. It is a conjugate prior for the precision matrix of a multivariate normal distribution. Its mean is $\nu V$.

The WinBUGS declaration dwish(R, nu) uses $R=V^{-1}$; the density convention is documented in the NIMBLE BUGS-language manual. For $\nu =p=3$ it is proper, while the expected inverse matrix does not exist: that expectation requires $\nu >p+1$. Specifying a Wishart prior on precision is therefore a substantive covariance assumption.