CLIME precision-matrix estimator Created 2026-10-03 Updated 2026-11-05
The CLIME precision-matrix estimator estimates a sparse matrix representing the precision matrix from a sample covariance matrix . An initial estimate solves
These are the entrywise matrix L1 norm and entrywise maximum norm, not the induced column-sum and row-sum norms. Writing for column and for the corresponding basis vector gives separate problems subject to , each a linear programming problem.[ref]
The column solutions need not form a symmetric matrix. The original construction selects the smaller-magnitude entry from each pair and copies it to both positions, using a common tie rule. Error analysis requires that the true precision matrix be feasible for the chosen tolerance.[ref]
Positive definiteness is not an optimization constraint here. For example, with and , the zero matrix is feasible and minimizes the objective. A positive-definite conclusion therefore needs further assumptions and a statistical theorem, rather than following from the program alone.
Graphical Lasso Created 2026-09-24 Updated 2026-11-05
The Graphical Lasso estimates a sparse matrix for the precision matrix by minimizing a penalized Gaussian objective over positive-definite matrices. With a sample covariance matrix and , the all-entry penalty convention is
The entrywise matrix L1 norm encourages zero entries, while the domain enforces positive definiteness. In a Gaussian graphical model, zero off-diagonal precision matrix entries encode conditional independence given the other coordinates.[ref]
The penalty convention matters. Writing , the Graphical-Lasso Karush-Kuhn-Tucker conditions give when diagonal entries are penalized, but when they are excluded.[ref] For example, diagonal and give under the all-entry convention.
The Graphical Lasso column regression relates its matrix optimization to Lasso subproblems. This is different from independently penalizing a raw covariance inverse, and the CLIME precision-matrix estimator uses a different constrained objective.
Write for a positive-definite matrix. Up to a constant independent of the parameters, the log-likelihood of the multivariate normal distribution is
The identity
shows that, for each , the unique optimizing mean is . Using the matrix trace to rewrite the first quadratic sum, the profile likelihood is
Therefore maximum likelihood estimation of the precision matrix, whenever the maximum exists, is exactly
This is the profile likelihood for a Gaussian precision matrix. The sample covariance matrix here uses divisor , as appropriate for maximum likelihood estimation.
There is an important rank qualification in the printed assumptions. If , differentiation gives , so the unique minimizer is . Uniqueness follows because the negative log-determinant is a strictly convex function on the positive-definite matrices. But full column rank of the uncentered alone does not imply : the rank of a centered sample covariance matrix depends on the centered design. For example, with and raw data matrix ,
Although has full column rank, setting gives objective . Thus an unpenalized maximizer need not exist under the literal raw-rank assumption. The likelihood reduction remains valid; existence requires full column rank of the centered data, equivalently . For the nonsingular Gaussian model this holds almost surely when .
Use the entrywise-penalty convention for the Graphical Lasso:
The sum is the entrywise matrix L1 norm. Its Graphical-Lasso Karush-Kuhn-Tucker conditions are
Here is symmetric; its nonzero-entry values are , and its zero-entry values are any numbers in . These are the subgradient optimality condition for this convex optimization problem. Since a positive-definite matrix has positive diagonal, and . Counting both off-diagonal entries in the penalty and in the trace makes the same appear in their stationarity equations. The common variant penalizing only off-diagonal entries instead sets ; the off-diagonal equations and the column argument below are unchanged.
For the given column problem, abbreviate
Its smooth part expands to plus a constant, because is symmetric and . The matrix is a positive-definite matrix, so this quadratic is a strictly convex function. Adding the convex L1 norm penalty preserves strict convexity, and the objective tends to infinity as . The minimizer exists and is unique. Its Karush-Kuhn-Tucker conditions are
We identify that minimizer directly from the precision matrix. Write and define
The denominator is positive. The off-diagonal part of column of gives
On the other hand, the off-diagonal Graphical-Lasso Karush-Kuhn-Tucker conditions read
Because , the coordinates of have the opposite signs from those of and vanish at exactly the same positions. The subgradient of the absolute value therefore satisfies
It follows that
These are exactly the Lasso column problem's Karush-Kuhn-Tucker conditions. Its unique minimizer must be , yielding
This is the Graphical Lasso column regression identity. The minus sign is essential: it converts the precision-entry subgradient into the regression-coefficient subgradient. The argument uses the inverse identity directly, so it does not require proving the supplied block inverse formula.