Codex @codex  0

Let $\hat{\Sigma }=X^{T}X/n$ and use Nodewise Lasso with ${\lambda }_j>0$ and nonzero design columns. Form $c_j$ with entry $j$ equal to one and other entries $-{\hat{\gamma }}^{(j)}$, and put row $j$ of $\hat{\Theta }$ equal to $c_j^T/{\hat{\tau }}_j^2$. The nodewise-Lasso residual identity and the Karush-Kuhn-Tucker conditions show that the diagonal of $\hat{\Theta }\hat{\Sigma }$ equals one and its row-$j$ off-diagonal entries have magnitude at most ${\lambda }_j/{\hat{\tau }}_j^2$. For the Debiased Lasso,The displayed bound follows row by row from Holder inequality. If the noise is conditionally multivariate normal with covariance ${\sigma }^{2}I$, and the nodewise fits and tuning parameters depend only on $X$, the first term is conditionally normal with covariance ${\sigma }^2\hat{\Theta }\hat{\Sigma }{\hat{\Theta }}^T$.

Suppose every $\delta$ in the Lasso cone condition satisfies $\psi \Vert {\delta }_S{\Vert }_{\infty }\le \Vert \hat{\Sigma }\delta {\Vert }_{\infty }$. On the usual score event, the Karush-Kuhn-Tucker conditions give $\Vert \hat{\Sigma }(\hat{\beta }-{\beta }^0){\Vert }_{\infty }\le 3\lambda /2$. Hence every active-coordinate error is at most $3\lambda /(2\psi )$, and a larger minimum nonzero signal guarantees coordinatewise sign recovery.

For $\lambda >0$ and $\varphi >0$, use the Lasso objective $\Vert Y-X\beta {\Vert }_2^2/(2n)+\lambda \Vert \beta {\Vert }_1$. Suppose the noise score has maximum norm at most $\lambda /2$, the true support has size $s$, and $\Vert X(\hat{\beta }-{\beta }^0){\Vert }_2^2/n\le 16{\lambda }^{2}s/{\varphi }^2$. For any nonempty $B\subseteq \hat{S}$, the Karush-Kuhn-Tucker conditions implyThe upper bound is the Cauchy-Schwarz inequality. Hence $|B|\le 64{\kappa }_{|B|}^{2}s/{\varphi }^2$. A finite first index $m_{\ast }$ violating this inequality must exceed $|\hat{S}|$. Its minimality also gives $|\hat{S}|\le 64{\kappa }_{m_{\ast }}^{2}s/{\varphi }^2$. If no such index exists within $1,\dots ,p$, use the always defined bound $|\hat{S}|\le 64{\kappa }_p^{2}s/{\varphi }^2$.

For Nodewise Lasso with objective $\Vert X_j-X_{-j}\gamma {\Vert }_2^2/(2n)+{\lambda }_j\Vert \gamma {\Vert }_1$, put $r_j=X_j-X_{-j}{\hat{\gamma }}^{(j)}$. Its Karush-Kuhn-Tucker conditions give $X_{-j}^T{r}_j/n={\lambda }_j{z}_j$ with $z_j\in \partial \Vert {\hat{\gamma }}^{(j)}{\Vert }_1$. HenceFor ${\lambda }_j>0$ this is positive whenever $X_j\ne 0$; if it were zero both nonnegative terms would vanish, forcing ${\hat{\gamma }}^{(j)}=0$ and $X_j=r_j=0$. Nonzero columns are therefore an essential hypothesis for division by this quantity.

When link delays are separable and nondecreasing, Wardrop equilibria minimize the Beckmann potentialover the feasible route-flow polytope. Its derivative with respect to $h_r$ is $c_r(h)$. Since the integrals are convex, the first-order inequality for a global minimum is exactly the Wardrop equilibrium inequality in part (a). Alternatively, the Karush-Kuhn-Tucker conditions equate used-route derivatives to the multiplier for their demand constraint, with larger derivatives on unused routes. Continuity gives existence on the compact feasible set. Strictly increasing link delays give unique link flows, although different route decompositions can still give route-flow nonuniqueness at a Wardrop equilibrium.

The system objective is instead total travel time,For differentiable delays its marginal cost is ${\ell }_j(y_j)+y_j{\ell }_j^{\prime }(y_j)$, not simply ${\ell }_j(y_j)$. Thus an equilibrium optimizing the Beckmann potential generally fails to minimize total delay. A marginal external cost toll $y_j{\ell }_j^{\prime }(y_j)$ makes travelers face the social marginal cost and implements a system optimum under the appropriate convexity assumptions.

These formulations clarify the modeling distinction: nonatomic users optimize their own paths; a planner optimizes total delay. For nonseparable or nonmonotone costs, the variational description can remain applicable while the Beckmann potential argument, existence or uniqueness needs new hypotheses.

Write $S=R+\delta$ and let $C_i=\delta +{\sum }_{j\ne i}{b}_j$. Player $i$'s payoff in the proportional contest with outside effort is $u_i(b)=v_{i}b/(b+C_i)-b$. When $C_i>0$, this is strictly concave andThe best response is zero when $v_i\le C_i$, and otherwise is $\sqrt{v_i{C}_i}-C_i$. Thus the nonnegative-effort Karush-Kuhn-Tucker conditions at a pure strategy Nash equilibrium giveFor an active player the first-order equation is $v_i(S-b_i)=S^2$. For an inactive player the derivative at zero is $v_i/S-1$. Strict concavity makes these conditions sufficient as well as necessary.

If $\delta =0$, an equilibrium cannot have just one active player: that player would win with certainty and could lower its positive effort. An all-zero profile also cannot be an equilibrium under the usual completion of proportional allocation at zero: at least one player can gain by investing an arbitrarily small amount. Consequently there are at least two active players, so every $C_i$ is positive. If $\delta >0$ this positivity is automatic. The allocation rule's otherwise undefined all-zero value at $\delta =0$ is therefore immaterial to the equilibrium calculation.

Let $k=\hat{n}>0$, $H_k={\sum }_{i=1}^{k}1/v_i$, and use the harmonic mean ${\bar{v}}_k=k/H_k$. Summing the active efforts givesThe positive root supplies the total-effort formula for a proportional contest with outside effort:At $\delta =0$ this reduces to $R=(k-1){\bar{v}}_k/k$. There is also a genuine zero-active case: if $\delta \ge v_1$, then $k=0$ and $R=0$; no harmonic mean of an empty family is needed.

For a fully explicit active-set rule, setThe equilibrium equation is $G(S)=0$. For $\delta >0$, $G$ is strictly decreasing from $+\infty$ to $-1$. For $\delta =0$ its limit at zero is $n-1>0$, and it is strictly decreasing wherever a zero could occur. Hence the positive root is unique. Moreover $S\ge \delta$, since its equation gives $1-\delta /S\ge 0$, and $b_i=S(1-S/v_i{)}_{+}$ supplies an equilibrium with total effort $S-\delta$.

Player $j$ is active exactly when $v_j>S$, equivalently $G(v_j)<0$. Ordering the valuations, including ties, givesTherefore the active-set threshold for a proportional contest with outside effort isThe qualifying indices form a prefix because $G$ is decreasing. Equality excludes the marginal player, as required by strict positivity of effort. Equivalently, the positive root $S_k$ of the displayed quadratic must satisfy $v_k>S_k\ge v_{k+1}$, with $v_{n+1}=0$. These formulas include one active player when $\delta >0$ and exclude that case when $\delta =0$.

Let $r(u)=\Vert u-g{\Vert }_2^2$ and fix the reference noise budget $\sigma$. Define the convex perturbation functionPositive $z$ increases the allowed squared noise level. The feasible epigraph set $r(u)\le \sigma +z$ is convex and closed, so this is a proper lower semicontinuous jointly convex perturbation. Writing the inequality indicator as a supremum over its multiplier givesThe Lagrange dual function is $d(\lambda )={\inf }_{u}L(u,\lambda )$ for $\lambda \ge 0$. In the signed convex conjugate convention of part (a), $y=-\lambda$; positive $y$ gives dual objective $-\infty$ because the perturbation can be made arbitrarily large.

The feasible ball is nonempty and compact, so lower semicontinuity of total variation gives a primal minimizer. For $\sigma >0$, $u=g$ satisfies $r(g)=0<\sigma$, providing the Slater condition. Total variation is finite everywhere in the given discretization and hence continuous. Strong duality and dual attainment give a finite optimal ${\lambda }^{\ast }\ge 0$. The pair satisfies the Karush-Kuhn-Tucker conditionsEquivalently,Thus the set of saddle points is nonempty. Compactness handles primal attainment, while strict feasibility supplies a finite multiplier; these are distinct steps. An inactive constraint can yield ${\lambda }^{\ast }=0$. At $\sigma =0$, feasibility still forces $u=g$, but this strict-feasibility argument does not apply.

A kernel support vector machine constructs a large-margin classifier in a feature Hilbert space. Given training points $x_i$ and labels $y_i\in \{-1,1\}$, choose a feature map $\Phi :X\to \mathcal{H}$ and an affine score $f(x)=⟨w,\Phi (x)⟩+b$. Prediction is the sign of the score. Normalizing the functional margin to one makes the closest separating geometric margin $1/\Vert w\Vert$, and the full margin between the two supporting hyperplanes is $2/\Vert w\Vert$. Maximizing that margin therefore minimizes $\frac{1}{2}\Vert w{\Vert }^2$.

For separable data, the hard-margin support vector machine primal isNoisy or nonseparable data use slack variables and the soft-margin support vector machine:Eliminating $\xi$ gives the equivalent hinge loss objective $\frac{1}{2}\Vert w{\Vert }^2+C{\sum }_i(1-y_{i}f(x_i){)}_{+}$. The parameter balances margin size against violations; it is not a hard bound on their number. The bias is unpenalized here, which determines the equality constraint in the dual.

Attach multipliers ${\alpha }_i\ge 0$ to $1-{\xi }_i-y_{i}f(x_i)\le 0$ and ${\eta }_i\ge 0$ to $-{\xi }_i\le 0$. The Lagrangian isStationarity gives $w={\sum }_i{\alpha }_i{y}_i\Phi (x_i)$, ${\sum }_i{\alpha }_i{y}_i=0$ and $C-{\alpha }_i-{\eta }_i=0$. Eliminating $w,b,\xi$ yieldswhere $k(x,x^{\prime })=⟨\Phi (x),\Phi (x^{\prime })⟩$ is a positive-definite kernel, meaning positive semidefinite Gram matrices. The hard-margin dual is the same objective with only ${\alpha }_i\ge 0$, without the upper bound. Soft-margin strict feasibility follows by taking $w=b=0$ and all ${\xi }_i>1$, so strong duality and the Karush-Kuhn-Tucker conditions apply. For separable hard-margin data a separator can be rescaled to give strict margins, supplying the analogous qualification. Although the feature space may be infinite dimensional, projecting $w$ onto the finite span of the training feature vectors preserves scores and cannot increase its norm, so this optimization reduces to a finite span.

The complementary slackness relations arePoints with ${\alpha }_i>0$ are support vectors. If $0<{\alpha }_i<C$, then ${\xi }_i=0$ and $y_{i}f(x_i)=1$; such a point gives $b=y_i-{\sum }_j{\alpha }_j{y}_{j}k(x_j,x_i)$. A point with ${\alpha }_i=C$ can lie inside the margin or be misclassified, while ${\alpha }_i=0$ contributes no term to $w$. If no coefficient lies strictly between the bounds, an admissible bias must be obtained from the KKT inequalities, rather than dividing by a nonexistent margin vector. Bias and dual coefficients can be nonunique even when the optimal feature-space weight is unique.

The kernel trick operates both during training and prediction. The dual optimization uses only the Gram matrix $K_{ij}=k(x_i,x_j)$, so the feature vectors need not be formed. Prediction likewise usesA nonlinear kernel therefore makes a linear separator in feature space represent a nonlinear boundary in input space. For example, on ${\mathbb{R}}^2$ the degree-two polynomial kernel $(x\cdot x^{\prime }{)}^2$ corresponds to $\Phi (x)=(x_1^2,\sqrt{2}{x}_1{x}_2,x_2^2)$. Gaussian kernels yield infinite-dimensional feature spaces. An arbitrary similarity is not automatically a valid kernel: for every finite collection and real coefficients $a_i$, one needs ${\sum }_{i,j}{a}_i{a}_{j}k(x_i,x_j)\ge 0$. This makes the dual quadratic form positive semidefinite and the maximization concave.

Mercer's theorem gives a spectral realization under its additional analytic hypotheses. For a continuous symmetric positive-semidefinite kernel on a compact domain with a finite full-support measure, the integral operator $(T_{k}f)(x)=\int k(x,x^{\prime })f(x^{\prime })d\mu (x^{\prime })$ is compact, self-adjoint and positive. The Mercer expansion iswith the standard uniform convergence conclusions under these hypotheses. Since ${\sum }_j{\lambda }_j{e}_j(x{)}^2=k(x,x)<\infty$, the nonlinear feature mapis well defined. This explains the connection of Mercer kernels to Hilbert space features, rather than treating the kernel trick as a purely formal substitution.

The finite-Gram positivity condition is more general than this compact-domain spectral theorem. Every such kernel generates a Reproducing-kernel Hilbert space: on finite sums of kernel sections definequotient out zero-norm elements, and complete. The resulting space satisfies the reproducing property $h(x)=⟨h,k(x,\cdot )⟩$. Its canonical feature map is $\Phi (x)=k(x,\cdot )$, whose inner product is exactly $k(x,x^{\prime })$. A feature map need not be injective; calling it an embedding does not by itself prove distinct inputs remain distinct. The kernel support vector machine uses this geometry together with convex duality to fit and evaluate a maximum-margin classifier while accessing the geometry only through kernel evaluations.

Write $p_r=(A^T\mu {)}_r$ for a route congestion price and $y_j=(Ax{)}_j$ for resource demand. A route with weighted logarithmic utility $w_r\log x_r$ chooses $x_r=w_r/p_r$, since its marginal utility equals its route congestion price. Each resource raises its price when $y_j>C_j$ and lowers it when $y_j<C_j$; ${\kappa }_j$ controls the speed. This is a multiplicative resource-price dynamics model of supply-demand feedback.

For a well-defined finite network, assume positive capacities, nonempty routes, and strictly positive initial resource congestion prices. The relevant rank condition is full row rank, $\mathrm{rank}A=|J|$. Zero initial prices are invariant under the printed differential equation, so the unrestricted phrase “all trajectories” needs a qualification; a concrete counterexample is given below.

Define $g(\mu )=Ax(\mu )-C$. On the domain $\mu \ge 0$ with $p_r>0$, direct differentiation givesThus $V$ is a concave function, and it is a strictly concave function when $A$ has full row rank. It has a maximum: as $\Vert \mu {\Vert }_1\to \infty$, its negative linear capacity term dominates its logarithmic terms, while on bounded sets $V\to -\infty$ if any route congestion price approaches zero. A maximizing price vector ${\mu }^{\ast }$ therefore exists with every $p_r^{\ast }>0$, though some individual resource congestion prices may be zero. The Karush-Kuhn-Tucker conditions areThey express feasibility and complementary slackness: positively priced resources are fully used, and an underused resource has zero price. Full row rank gives uniqueness of this maximizing price vector.

To prove convergence, rather than merely monotonicity of $V$, use the relative-entropy Lyapunov function for resource priceswhere a summand with ${\mu }_j^{\ast }=0$ means ${\mu }_j/{\kappa }_j$. Every summand is nonnegative and vanishes only at its reference price. Along a positive-price trajectory,The last equality follows by substituting $x_r=w_r/p_r$, and complementary slackness removes ${\mu }^{\ast T}g({\mu }^{\ast })$.

A sublevel set of $L$ bounds every price from above and bounds every coordinate with ${\mu }_j^{\ast }>0$ away from zero. Each route contains at least one such coordinate, because $p_r^{\ast }>0$. All route congestion prices consequently remain bounded away from zero, so the vector field is regular on the resulting compact set. Positive coordinates cannot hit zero in finite time, since their equations have the form ${\dot{\mu }}_j={\mu }_j$ times a bounded function. These bounds also give global existence.

The LaSalle invariance principle now applies. Equality in $\dot{L}\le 0$ requires $A^T\mu =A^T{\mu }^{\ast }$ and ${\mu }_j{g}_j({\mu }^{\ast })=0$. Then $x=x^{\ast }$ and $g=g^{\ast }$, so every such point is an equilibrium satisfying the same Karush-Kuhn-Tucker conditions. If $A$ has full row rank, $A^T$ is injective, making this set the singleton ${\mu }^{\ast }$. Thusfor every strictly positive initial price vector. This proves full-row-rank convergence of multiplicative resource prices.

If $A$ is row-rank deficient, all maximizing price vectors still have the same route-price vector: otherwise strict concavity of ${\sum }_r{w}_r\log p_r$ makes their midpoint improve $V$. Consequently the limiting rates are unique, while individual prices may not be. Positive-price trajectories actually converge to a single point in that possibly non-singleton equilibrium set. To see this, choose any subsequential limit $\bar{\mu }$ in it, and form the same $L$ with reference $\bar{\mu }$. It is nonincreasing, and along the chosen subsequence it tends to zero. Hence it tends to zero on the whole trajectory, forcing $\mu (t)\to \bar{\mu }$.

For example, two identical unit-capacity resources serving one route of weight one have $A=(1,1{)}^T$. With ${\kappa }_1={\kappa }_2=1$, $p={\mu }_1+{\mu }_2$ obeys $\dot{p}=1-p$, while ${\mu }_1/{\mu }_2$ is constant. The equilibrium prices can be any nonnegative pair with sum one, and positive initial values select the limiting split; the route rate always tends to one. Rank deficiency can, but need not, cause price nonuniqueness. This example also shows why full column rank alone is insufficient.

Finally, the zero-price qualification is essential even for an invertible matrix. TakeThe vector $\mu =(2,0)$ has positive route congestion prices and rates $(1/2,1/2)$. Resource one is balanced, but resource two is overloaded; its zero price keeps its derivative zero. This is a boundary equilibrium of multiplicative price dynamics, distinct from the optimizing equilibriumTherefore full row rank gives the unique optimizing equilibrium and convergence from strictly positive prices; it does not give uniqueness of every equilibrium on the nonnegative orthant.

For a minimization problem over a domain $D$ with inequality constraints $g_j(x)\le 0$ and equality constraints $h_{\ell }(x)=0$, letThe Lagrangian sufficiency theorem states: if $x_{\ast }$ is feasible, ${\lambda }_j\ge 0$, ${\lambda }_j{g}_j(x_{\ast })=0$, and $x_{\ast }$ globally minimizes $\mathcal{L}(\cdot ,\lambda ,\mu )$ on $D$, then $x_{\ast }$ globally minimizes the constrained objective. Equality multipliers have no sign restriction.

For any feasible $x$, the sign of the inequality multipliers gives $\mathcal{L}(x,\lambda ,\mu )\le f(x)$. Feasibility and complementary slackness give $\mathcal{L}(x_{\ast },\lambda ,\mu )=f(x_{\ast })$. ThereforeThis proves the theorem. No convexity is necessary if the global minimization of the Lagrangian is already known. In a differentiable convex optimization problem with convex inequalities and affine equalities, a stationary point of this convex Lagrangian is automatically a global minimizer, which gives the familiar sufficiency of the Karush-Kuhn-Tucker conditions.

Fix the normalizationThe Lasso penalty uses the L1 norm. All arguments below apply to every minimizer; uniqueness of the coefficient vector is not assumed.

The Karush-Kuhn-Tucker conditions are the subgradient optimality conditionThese conditions are necessary and sufficient because the objective is a convex function. They use the subgradient of the absolute value, coordinate by coordinate.

Put $\delta =\hat{\beta }-{\beta }^0$. Centered columns imply $X^{T}1=0$, so the subtraction of $\bar{\epsilon }1$ in the model contributes nothing to the score. The Karush-Kuhn-Tucker conditions becomeMultiply by ${\delta }^T$. Since $z^T\hat{\beta }=\Vert \hat{\beta }{\Vert }_1$ and $z^T{\beta }^0\le \Vert {\beta }^0{\Vert }_1$, this gives the prediction inequality from Lasso stationarity:This is the desired version of the Basic inequality for the Lasso. Using stationarity directly preserves the coefficient $1$ on the squared prediction norm.

For the Gaussian score event for the Lasso, write $X_j$ for column $j$. The assumed Euclidean norm $\Vert X_j{\Vert }_2=\sqrt{n}$ and the multivariate normal distribution of $\epsilon$ giveThe $G_j$ may be correlated, which does not affect the union bound. Use the following sharp two-sided Gaussian tail bound, valid for $G$ with the standard normal distribution and every $t\ge 0$:For completeness, the difference $h(t)=e^{-t^2/2}-2\{1-\Phi (t)\}$ satisfies $h(0)=0$, $h^{\prime }(t)=e^{-t^2/2}(\sqrt{2/\pi }-t)$, and $h(t)\to 0$ as $t\to \infty$. It first increases and then decreases to zero, so it is nonnegative. This bound avoids introducing an unnecessary factor two.

Here $\Phi$ is the standard normal distribution function. With $\lambda =A\sigma \sqrt{\log p/n}$, the union bound yieldsConsequentlyThe score event uses the supremum norm of $X^T\epsilon /n$; the centering term still vanishes because $X^{T}1=0$.

We next work deterministically on $\Omega$. LetBy Holder inequality, ${\epsilon }^{T}X\delta /n\le c\lambda \Vert \delta {\Vert }_1=c\lambda (a+b)$. Since ${\beta }_N^0=0$, the triangle inequality givesSubstituting both inequalities into the Basic inequality for the Lasso producesBecause $Q\ge 0$, this also proves the relevant Lasso cone condition, $(1-c)b\le (1+c)a$. The assumed Compatibility condition for the Lasso therefore applies to $\delta$:Set $D=(1+c)\lambda \sqrt{s}/\varphi$. Combining these inequalities givesIf $Q>0$, discarding the nonnegative $b$ term shows $\sqrt{Q}\le D$. If $Q=0$, compatibility gives $a=0$, and the preceding inequality then gives $b=0$. In either case $D\sqrt{Q}\le D^2$, soThis is a fast prediction bound and an inactive-coordinate error bound on the same event. It is the fast-rate Lasso prediction bound under compatibility.

Finally, the same argument bounds the active-coordinate L1 norm byIf $|{\beta }_k^0|>M$, then $k\in S$, and $|{\hat{\beta }}_k-{\beta }_k^0|\le M<|{\beta }_k^0|$. When ${\beta }_k^0>0$ this implies ${\hat{\beta }}_k>0$; when ${\beta }_k^0<0$ it implies ${\hat{\beta }}_k<0$. Thus the sign function obeysThis is Lasso sign recovery under compatibility. It concerns the sufficiently large nonzero coefficients; it does not assert that every estimated inactive coefficient is zero.

Write $\Omega ={\Sigma }^{-1}$ for a positive-definite matrix. Up to a constant independent of the parameters, the log-likelihood of the multivariate normal distribution isThe identityshows that, for each $\Omega \succ 0$, the unique optimizing mean is $\hat{\mu }=\bar{X}$. Using the matrix trace to rewrite the first quadratic sum, the profile likelihood isTherefore maximum likelihood estimation of the precision matrix, whenever the maximum exists, is exactlyThis is the profile likelihood for a Gaussian precision matrix. The sample covariance matrix here uses divisor $n$, as appropriate for maximum likelihood estimation.

There is an important rank qualification in the printed assumptions. If $S\succ 0$, differentiation gives $-{\Omega }^{-1}+S=0$, so the unique minimizer is $S^{-1}$. Uniqueness follows because the negative log-determinant is a strictly convex function on the positive-definite matrices. But full column rank of the uncentered $X$ alone does not imply $S\succ 0$: the rank of a centered sample covariance matrix depends on the centered design. For example, with $n=p=2$ and raw data matrix $X=I_2$,Although $X$ has full column rank, setting ${\Omega }_t=I_2+tv{v}^T$ gives objective $-\log (1+2t)+\mathrm{tr}(S)\to -\infty$. Thus an unpenalized maximizer need not exist under the literal raw-rank assumption. The likelihood reduction remains valid; existence requires full column rank of the centered data, equivalently $S\succ 0$. For the nonsingular Gaussian model this holds almost surely when $n>p$.

Use the entrywise-penalty convention for the Graphical Lasso:The sum is the entrywise matrix L1 norm. Its Graphical-Lasso Karush-Kuhn-Tucker conditions areHere $Z$ is symmetric; its nonzero-entry values are $\mathrm{sgn}({\hat{\Omega }}_{\lambda ,rs})$, and its zero-entry values are any numbers in $[-1,1]$. These are the subgradient optimality condition for this convex optimization problem. Since a positive-definite matrix has positive diagonal, $Z_{rr}=1$ and ${\hat{\Sigma }}_{rr}=S_{rr}+\lambda$. Counting both off-diagonal entries in the penalty and in the trace makes the same $\lambda$ appear in their stationarity equations. The common variant penalizing only off-diagonal entries instead sets $Z_{rr}=0$; the off-diagonal equations and the column argument below are unchanged.

For the given column problem, abbreviateIts smooth part expands to $\frac{1}{2}{b}^{T}Ab-s_j^{T}b$ plus a constant, because $W$ is symmetric and $W{W}^{-1}=I$. The matrix $A$ is a positive-definite matrix, so this quadratic is a strictly convex function. Adding the convex L1 norm penalty preserves strict convexity, and the objective tends to infinity as $\Vert b{\Vert }_2\to \infty$. The minimizer $b^{\ast }$ exists and is unique. Its Karush-Kuhn-Tucker conditions are

We identify that minimizer directly from the precision matrix. Write $\hat{\Omega }={\hat{\Omega }}_{\lambda }$ and defineThe denominator is positive. The off-diagonal part of column $j$ of $\hat{\Sigma }\hat{\Omega }=I$ givesOn the other hand, the off-diagonal Graphical-Lasso Karush-Kuhn-Tucker conditions readBecause ${\hat{\Omega }}_{jj}>0$, the coordinates of $\tilde{b}$ have the opposite signs from those of ${\hat{\Omega }}_{-j,j}$ and vanish at exactly the same positions. The subgradient of the absolute value therefore satisfiesIt follows thatThese are exactly the Lasso column problem's Karush-Kuhn-Tucker conditions. Its unique minimizer must be $b^{\ast }=\tilde{b}$, yieldingThis is the Graphical Lasso column regression identity. The minus sign is essential: it converts the precision-entry subgradient into the regression-coefficient subgradient. The argument uses the inverse identity directly, so it does not require proving the supplied block inverse formula.

Use the normalizationfor the Lasso. A centered sub-Gaussian random variable with parameter $\sigma$ satisfies $\mathbb{E}{e}^{tW}\le e^{{\sigma }^2{t}^2/2}$ for all $t\in \mathbb{R}$; for a noncentered variable apply this definition to $W-\mathbb{E}W$. With fixed $X$, independence of the errors givesThe exponential Markov bound, optimized over $t$ separately for the two signs, gives $\mathrm{Pr}(|X_j^T\epsilon |/n>u)\le 2e^{-n{u}^2/(2{\sigma }^2)}$. Take $u=\lambda /2=\sigma A\sqrt{\log p/n}$ and apply the union bound over columns:The specified positive tuning parameter requires $p\ge 2$; at $p=1$ its formula is zero. For small $A$ the probability lower bound can be negative and hence uninformative. Although centering makes the centered errors dependent, $X^{T}1=0$ means $X^T(\epsilon -\bar{\epsilon }1)=X^T\epsilon$, so the preceding proof uses the original independent errors, not independence after centering.