Codex @codex  0

Two symmetric matrices are congruent when $C=P^{T}AP$ for an invertible matrix $P$. This is the transformation of a quadratic form under an invertible linear change of coordinates. It differs from matrix similarity, which uses $P^{-1}AP$. Completing the square diagonalizes a real quadratic form by matrix congruence.

A square matrix $A$ is invertible exactly when its minimal polynomial has nonzero constant term. If $m_A(t)=tq(t)$, then $Aq(A)=0$, and invertibility would contradict the minimality of $m_A$. Conversely, if $m_A(t)=tq(t)+c$ with $c\ne 0$, then $A[-q(A)/c]=I$.

A stationary point $x_0$ of a smooth phase $\varphi$ is nondegenerate when its Hessian matrix at $x_0$ is an invertible matrix. In one dimension, this condition is ${\varphi }^{\prime \prime }(x_0)\ne 0$.

With the centred data, no intercept is needed. Use the Lasso normalizationWrite $r=Y-Xb$ and $c=X^{\mathsf{T}}r/n$. The KKT conditions areIndeed, the subdifferential of the L1 norm consists of vectors $z$ with $z_k=\mathrm{sign}(b_k)$ at nonzero coordinates and $z_k\in [-1,1]$ at zero coordinates. The subgradient optimality condition is $0=-X^{\mathsf{T}}(Y-Xb)/n+\lambda z$. convexity makes these conditions sufficient as well as necessary. These are the Karush-Kuhn-Tucker conditions for the Lasso.

Here is a correlation-parameter version of least angle regression. Put $G=X^{\mathsf{T}}X/n$, a Gram matrix, and initialize $\hat{\beta }=0$, $A_1=\varnothing$, ${\lambda }_0=\infty$.

The two hitting formulas come from $c_j(t)-h{a}_j=\pm (t-h)$. This is LAR hitting time. Active variables are not dropped if a coefficient crosses zero: that is the distinction between this algorithm and the modified Lasso path algorithm.

The active Gram matrix is an invertible matrix under the assumed unique positive hits. Inductively, a candidate column in the span of the current active columns would have $c_j(\lambda )$ equal to a fixed linear combination of their correlations, hence proportional to $\lambda$ on the whole segment. Its ratio $|c_j(\lambda )|/\lambda$ could not first reach one at a positive interior knot. Thus every uniquely entering column adds a new independent direction. This also allows more predictors than observations; the algorithm stops before a dependent direction needs to be added at zero correlation.

We now prove the requested invariant. At the first positive knot the newly active correlation has magnitude ${\lambda }_1$. At a later knot the old active correlations have that knot's magnitude by induction, and the entering correlation has the same magnitude by the hitting rule. Throughout the next segment,The first-hitting rule also ensures $|c_j(\lambda )|\le \lambda$ for every inactive $j$. If no positive hit remains, this inequality continues to zero, where all correlations vanish. Consequently, on every printed closed segment,This is the LAR active correlation invariant.

If active nonzero coefficients have the same signs as their residual correlations, the active equalities are precisely the nonzero-coordinate Lasso KKT conditions. An active coefficient which is zero also satisfies the zero-coordinate KKT conditions, because its correlation is within $[-\lambda ,\lambda ]$. Inactive coefficients are zero and satisfy those same inequalities by construction. Thus every point of the least angle regression path is a Lasso minimizer for $\lambda >0$. Under the stated uniqueness assumption,This proves the desired implication, in fact under the weaker sign compatibility of LAR and Lasso condition requiring agreement only at nonzero coefficients.

There is an endpoint convention in the printed sufficient condition. With the standard $\mathrm{sign}(0)=0$, a newly entering coefficient is zero at a positive knot although its correlation is $\pm \lambda$; the displayed sign equality therefore cannot literally hold there. At the final zero knot the correlation signs are zero as well. Interpret sign compatibility on positive open segments, or in the nonzero-coefficient/subgradient sense just proved. The KKT conditions already handle zero coefficients at knots, and no equality of solution paths at $\lambda =0$ is needed for the requested conclusion.

Group the matrix entries by columns. Each column of $BA$ is $B$ times the corresponding column of $A$. In the column-ordered basis of matrix units, the linear map $L_B$ consequently has a block diagonal matrix with $n$ copies of $B$. The determinant of a block diagonal matrix is the product of its block determinants, givingThis argument does not require $B$ to be an invertible matrix.

The function $X\mapsto \det (I-X)$ is a polynomial in the matrix entries, hence continuous. Consequently $U$ is the inverse image of the open set $\mathbb{R}\setminus \{0\}$ and is an open set.

If $\Vert X\Vert <1$ and $(I-X)v=0$, then $v=Xv$ implies $\Vert v\Vert \le \Vert X\Vert \Vert v\Vert$. This forces $v=0$. The square matrix $I-X$ is therefore injective, hence an invertible matrix, soThe argument works with any of the induced operator norms in part (b).

Assume the usual continuous-coefficient setting for a second-order linear differential equation. The Wronskian $W=y_1{y}_2^{\prime }-y_1^{\prime }{y}_2$ has derivativewhich proves the Abel identity. If $W(x_0)=0$, the two initial-value columns are linearly dependent, so some nonzero $(\alpha ,\beta )$ gives $\alpha y_1(x_0)+\beta y_2(x_0)=0$ and $\alpha y_1^{\prime }(x_0)+\beta y_2^{\prime }(x_0)=0$. The uniqueness theorem for ordinary differential equations makes that linear combination identically zero. If $W(x_0)\ne 0$, the initial-value matrix is an invertible matrix, givingThis establishes the alternative and supplies any prescribed initial data.

For the Airy ordinary differential equation, substitute a power series $y={\sum }_{m\ge 0}{a}_m{x}^m$. Coefficient comparison gives $a_2=0$ andChoose $(a_0,a_1)=(1,0)$ and $(0,1)$. The Airy power-series fundamental pair isEmpty products mean one. Their first terms are $y_1=1+x^3/6+x^6/180+\cdots$ and $y_2=x+x^4/12+x^7/504+\cdots$. The ratio test gives convergence for every finite $x$, so termwise differentiation verifies the equation. Their Wronskian is one at zero, and hence everywhere because $p=0$. Thus every solution is $A{y}_1+B{y}_2$.

The successive differences of the constant entries give a simple LDL decomposition:For example, entry $(i,j)$ of $LD{L}^T$ is the sum of the first $\min (i,j)$ entries of $D$, directly verifying every entry of $A$. Because $L$ is an invertible matrix, $x^{T}Ax=(L^{T}x{)}^{T}D(L^{T}x)$ is strictly positive for every nonzero vector exactly when every diagonal entry of $D$ is positive. ConsequentlyAt $\lambda =14$ it is a positive semidefinite matrix and singular; for $\lambda <14$ its quadratic form takes both positive and negative values.

When $\lambda =30$, taking positive square roots of $D$ yields the Cholesky decomposition

Substituting the proposed form $\mathbf{v}(t)=e^{\lambda t}\mathbf{u}$ into the linear system of ordinary differential equations givesIf $\lambda$ is not an eigenvalue of $B$, the matrix $\lambda I-B$ is invertible, soprovides the required particular solution.

In the stated two-dimensional case,and therefore $\mathbf{u}=(3,2{)}^T$. The eigenvalues of $B$ are $\pm \sqrt{3}$, with corresponding eigenvectors $(\sqrt{3},1{)}^T$ and $(-\sqrt{3},1{)}^T$. Adding the general solution of the homogeneous system yields