Precision matrix ($\Omega ={\Sigma }^{-1}$)

The precision matrix is the inverse of a nonsingular covariance matrix. For a multivariate normal distribution, a zero off-diagonal precision entry encodes conditional independence of the corresponding coordinates given all remaining coordinates.

For a nonsingular multivariate normal distribution with precision matrix $\Omega$, fixing all coordinates except $X_i$ leaves a density proportional to $\exp [-{\Omega }_{ii}(x_i-m_i{)}^2/2]$ after completing the square. Thus its conditional variance is the reciprocal diagonal precision entry. This is conditioning on every other coordinate, not the marginal variance $({\Omega }^{-1}{)}_{ii}$.

In a Gaussian graphical model, regressing component $k$ on the others gives the displayed coefficients, residual variance $1/{\Omega }_{kk}$ and intercept ${\mu }_k-{\mu }_{-k}^{\mathsf{T}}{\theta }_{-k,k}$. Opposite directions have identical zero patterns and signs because $\Omega$ is symmetric with positive diagonal, but generally unequal coefficient magnitudes.

For independent multivariate normal observations with unknown mean and nonsingular covariance, profiling the mean at its sample value gives negative log-likelihood proportional to $-\log \det \Omega +\mathrm{tr}(S\Omega )$ over $\Omega \succ 0$. The sample covariance matrix uses divisor $n$. Its minimizer exists exactly when $S\succ 0$, and is then $S^{-1}$.

If a positive-definite matrix $Q=L{L}^T$ is the precision matrix of a multivariate normal distribution, draw independent $\xi \sim N(0,I)$ and solve $L^T\eta =\xi$. Then $m+\eta$ has covariance $L^{-T}{L}^{-1}=Q^{-1}$. This avoids forming the inverse and exploits a banded Cholesky decomposition when $Q$ is banded.

The constrained ${\ell }^1$ minimization estimator minimizes $\Vert \Omega {\Vert }_1$ subject toIts constraints and objective separate by columns. Feasibility of the true precision matrix and elementary matrix-norm inequalities yield entrywise error bounds under an entrywise covariance-estimation bound.

The graphical Lasso estimates a sparse precision matrix by minimizing a Gaussian negative log-likelihood plus an entrywise ${\ell }^1$ penalty.

For a Graphical Lasso solution with inverse $\hat{\Sigma }$, the unique minimizer of $\frac{1}{2}\Vert Wb-W^{-1}{S}_{-j,j}{\Vert }_2^2+\lambda \Vert b{\Vert }_1$, $W^2={\hat{\Sigma }}_{-j,-j}\succ 0$, is $b=-{\hat{\Omega }}_{-j,j}/{\hat{\Omega }}_{jj}$. The inverse identity gives ${\hat{\Sigma }}_{-j,j}={\hat{\Sigma }}_{-j,-j}b$, and the off-diagonal Graphical-Lasso Karush-Kuhn-Tucker conditions become the column Lasso conditions because the corresponding signs are opposite.

For objective $-\log \det \Omega +\mathrm{Tr}(S\Omega )+\lambda \Vert \Omega {\Vert }_{1,\mathrm{entry}}$, the KKT conditions are $-{\Omega }^{-1}+S+\lambda Z=0$ with $Z_{ij}\in \partial |{\Omega }_{ij}|$.