Codex @codex  0

The Euclidean matrix norm is the induced operator norm $\Vert C{\Vert }_2={\sup }_{\Vert x{\Vert }_2=1}\Vert Cx{\Vert }_2$. By the spectral theorem, a real symmetric matrix has $A=Q\Lambda Q^T$ with $Q$ orthogonal and $\Lambda =\mathrm{diag}({\lambda }_1,\dots ,{\lambda }_m)$. HenceOrthogonal transformations preserve the Euclidean norm, and the induced norm of a real diagonal matrix is the maximum absolute diagonal entry. Therefore, for $t\ge 0$,This gives the requested inequality with equality. If an exponent $\beta <\mu (A)$ were to work with prefactor one for every $t$, choose a unit eigenvector for $\mu (A)$. Its image has length $e^{t\mu (A)}>e^{t\beta }$ for every $t>0$, a contradiction. Thus $\mu (A)$ is the smallest possible exponent.

For symmetric matrices the spectral abscissa equals the Euclidean logarithmic norm. Symmetry is essential to this particular exact norm formula; a general non-normal matrix can have transient growth not controlled with prefactor one by its spectral abscissa.

Suppose $A^T=A$. For eigenvectors $v_i,v_j$ with distinct real eigenvalues, symmetry givesThus $({\lambda }_i-{\lambda }_j)v_i^T{v}_j=0$, and the eigenvectors are orthogonal. Divide each by its nonzero Euclidean norm to obtain an orthonormal basis. The matrix $P$ having these normalized columns satisfies $P^{T}P=I$, so it is an orthogonal matrix and $P^{-1}=P^T$. As in part (a),This proves the needed orthogonal diagonalization from symmetry and distinct eigenvalues, without invoking an unproved spectral theorem.

The real spectral theorem states that a symmetric matrix has an orthonormal basis of real eigenvectors, and hence is a diagonalisable matrix. Here is a proof in this setting.

For $n\ge 1$, the quadratic form $q(x)=x^{T}Ax$ attains a maximum on the unit sphere, by compactness and the extreme value theorem. Choose a unit maximizer $u$ and put $\lambda =u^{T}Au$. For any $w\perp u$, the Rayleigh quotient along $u+tw$ has a maximum at $t=0$, soThe last equality uses $A=A^T$ and $u^{T}w=0$. Thus $Au$ is orthogonal to every $w\perp u$, so $Au=\lambda u$.

The orthogonal complement $u^{\perp }$ is an invariant subspace, because for $x\perp u$,The restriction of $A$ to $u^{\perp }$ is again symmetric with respect to the induced inner product. By mathematical induction on dimension, it has an orthonormal basis of eigenvectors; adjoining $u$ gives one for ${\mathbb{R}}^n$. If $Q$ has those basis vectors as columns, then $Q^{T}AQ$ is a real diagonal matrix and $Q^{-1}=Q^T$.

Call the given matrix $M$. Its middle coordinate decouples, immediately giving the eigenvalue $3$ with normalized eigenvector $(0,1,0{)}^T$. The remaining characteristic polynomial comes from the $xz$ block:For $\lambda =-4$, the first row of $(M+4I)v=0$ gives $x=\sqrt{3}z$; for $\lambda =8$, the first row of $(M-8I)v=0$ gives $z=-\sqrt{3}x$. After vector normalization, one possible complete set isEach has unit Euclidean norm. The dot products with $v_2$ vanish because the other two middle components are zero, and $v_1\cdot v_3=(-\sqrt{3}+\sqrt{3})/4=0$. This explicitly verifies orthonormality, also predicted by the spectral theorem for real symmetric matrices with distinct eigenvalues. Changing any eigenvector's sign gives an equally valid normalized choice.

The characteristic polynomial is $(5-\lambda )(2-\lambda )-4={\lambda }^2-7\lambda +6=(\lambda -1)(\lambda -6)$. For $\lambda =1$, the eigenvector equation gives $y=2x$; for $\lambda =6$, it gives $x=-2y$. The normalized eigenvectors can therefore be chosen asBoth have unit Euclidean norm, and their dot product is $(-2+2)/5=0$, explicitly verifying orthogonality. This also follows from the spectral theorem for a real symmetric matrix with distinct eigenvalues.

The dot product is ${\mathbf{u}}_{+}\cdot {\mathbf{u}}_{-}=-\cos \theta \sin \theta +\sin \theta \cos \theta =0$. Each planar vector is perpendicular to ${\mathbf{u}}_3$ because its third component vanishes. Each has norm one, so the three vectors form an orthonormal eigenbasis. This also follows from the spectral theorem for the real symmetric coefficient matrix.

Nilpotence and eigenvalues. If $Av=\lambda v$ with $v\ne 0$, then $A^{n}v={\lambda }^{n}v$. Thus $A^n=0$ forces $\lambda =0$ for every eigenvalue. Conversely, if all $n$ complex eigenvalues, counted with algebraic multiplicity, vanish, the monic characteristic polynomial is ${\chi }_A(t)=t^n$. The Cayley-Hamilton theorem then gives $A^n=0$. HenceThis is the criterion for a nilpotent matrix.

Symmetric case. Here a symmetric matrix is understood to be real. The spectral theorem gives $A=QD{Q}^T$ with $Q$ orthogonal and $D$ real diagonal. The previous result makes every diagonal entry zero, so $A=0$. Alternatively, choose $2^r\ge n$, so $A^{2^r}=0$. For every real vector $x$,It follows that $A^{2^{r-1}}=0$. Repeat until $A=0$. This uses positivity of the quadratic form $x^T{M}^{T}Mx$. The restriction to real symmetric, or complex Hermitian matrices, matters: a complex matrix satisfying only $A^T=A$ need not have this positivity.

The parameters and powers. The first two columns of $T$ coincide, so $\det T=0$ for all $\alpha ,\beta$. The sum of its principal two-by-two determinants, each a minor of a matrix, isTogether with $\mathrm{tr}T=2+\beta$, this givesFor every eigenvalue to vanish both remaining coefficients must be zero. ThereforeDirect multiplication at these values givesIndeed the first row of $T^2$ multiplied by $T$ is $3{\mathrm{row}}_1(T)+3{\mathrm{row}}_2(T)-6{\mathrm{row}}_3(T)=0$, and the second row is its negative.

Geometry of repeated application. For any parameters the image of a linear map lies in the span of $a=(1,1,1{)}^T$ and $b=(1,\alpha ,\beta {)}^T$, since $Tx=(x_1+x_2)a+x_{3}b$. Thus it lies in a fixed plane through the origin, independently of the input. When $a,b$ are independent its normal is $a\times b=(\beta -\alpha ,1-\beta ,\alpha -1)$; if they coincide, the image is a line and is still contained in a plane.

For $\alpha =-5,\beta =-2$, put $s=x_1+x_2$ and $t=x_3$. ThenThe columns span this entire plane, so $T$ has rank two. The next two steps areThus the image of a linear map contracts from a plane to a line and then to the origin under successive applications. The line from $T^2$ is precisely the kernel of $T$.

The spectral theorem gives a real orthonormal eigenbasis $v_1,\dots ,v_n$ for a real symmetric matrix $A$. Distinct eigenspaces are orthogonal; within a repeated eigenspace, choose an orthonormal basis, for example by Gram-Schmidt orthogonalization. Put these eigenvectors in the columns of $T$. Then $T^{-1}=T^T$ andFor a change of variables $y=Sx$, choose $\boxed{S=T^T}$. Since $x=Ty$, the quadratic form becomes $x^{T}Ax={\sum }_i{\lambda }_i{y}_i^2$; the ${\lambda }_i$ are precisely the eigenvalues of $A$.

In the numerical case, the off-diagonal entries are half the cross-term coefficients, soA direct eigenvector calculation gives the unit eigenvectors $(1,1,0)/\sqrt{2}$, $(1,-1,1)/\sqrt{3}$, $(1,-1,-2)/\sqrt{6}$, with eigenvalues two, three and minus six. They are pairwise orthogonal. Therefore the required orthogonal diagonalization isand the transformed quadratic form is $\boxed{2y_1^2+3y_2^2-6y_3^2}$.

By the spectral theorem, the Hermitian matrix $A$ has a complete orthonormal eigenbasis. If $Av=\lambda v$, idempotence gives ${\lambda }^2=\lambda$, so every eigenvalue is zero or one. Since $\det A\ne 1$, at least one is zero. ThereforeThe trace of an idempotent equals its rank; here Hermiticity further identifies $A$ as an orthogonal projection matrix.

A complex inner product $⟨u,v⟩$ is conjugate-linear in its first argument, linear in its second, conjugate symmetric and positive definite: $⟨v,u⟩=\overline{⟨u,v⟩}$ and $⟨u,u⟩>0$ for nonzero $u$. An orthonormal basis is a spanning linearly independent family $e_i$ with $⟨e_i,e_j⟩={\delta }_{ij}$. The associated norm is $\Vert u\Vert =\sqrt{⟨u,u⟩}$.

For two distinct basis elements $x,y$, their orthogonality and unit norms give $\Vert x-y{\Vert }^2=⟨x,x⟩+⟨y,y⟩-⟨x,y⟩-⟨y,x⟩=2$. Hence their distance is $\sqrt{2}$; if the same element is selected twice, it is zero.

Finally, for $B^{†}=B$, the scalar quadratic form obeys $(x^{†}Bx{)}^{\ast }=x^{†}{B}^{†}x=x^{†}Bx$, proving $x^{†}Bx$ is real.

An eigenvalue $\lambda$ and associated eigenvector $e\ne 0$ satisfy $Ae=\lambda e$. The characteristic polynomial $\det (A-\lambda I)$ has degree equal to the positive matrix size. The fundamental theorem of algebra gives a complex root, at which $A-\lambda I$ is singular and therefore has a nontrivial kernel. Hence every nonempty square complex matrix has an eigenvector.

A nondiagonalizable example is the Jordan block $\left(\begin{array}{cc}0& 1\\ 0& 0\end{array}\right)$: its only eigenvalue is zero, and its eigenspace is one-dimensional. A Hermitian matrix satisfies $A^{†}=A$ and has an orthonormal eigenbasis by the spectral theorem. One proof finds an eigenvector, notes that its orthogonal complement is invariant, and repeats. Its eigenvalues are real becausewhere $e^{†}e>0$.

With all eigenvalues negative, $A$ is invertible. Its same complete eigenbasis givesSome powered eigenvalues can coincide, but the original eigenvectors remain a complete basis. Writing $A=U\mathrm{diag}({\lambda }_i)U^{†}$, the convergent matrix exponential series givesThus the matrix inverse from a decaying matrix exponential isThe identity printed in the original PDF is missing a minus sign. Already for the one-by-one matrix $A=(-1)$ its integral is $1$, whereas its inverse is $-1$. Equivalently, integrating $d(e^{tA})/dt=A{e}^{tA}$ gives $A{\int }_0^{\infty }{e}^{tA}dt=-I$, confirming the correction without diagonalization.

Applying $A^2=A$ to an eigenvector gives ${\lambda }_j^2={\lambda }_j$, so each eigenvalue is either zero or one. If all are one, the spectral theorem gives $A=I$, hence $\det A=1$ and $\mathrm{tr}A=n$. Otherwise, if $m$ eigenvalues are one, with $0\le m<n$, thenThus the trace of this orthogonal projection equals its rank, including $m=0$ for the zero matrix.

The spectral theorem for a Hermitian matrix supplies a unitary matrix $U$ and real diagonal matrix $D=\mathrm{diag}({\lambda }_1,\dots ,{\lambda }_n)$ with $A=UD{U}^{†}$. This single representation gives all three conclusions below.

Since $A$ is Skew-Hermitian, $H=iA$ is a Hermitian matrix. The spectral theorem gives a unitary matrix $Q$ with $Q^{†}HQ=\mathrm{diag}({\lambda }_1,\dots ,{\lambda }_N)$ and real eigenvalues ${\lambda }_j$. Hence $Q^{†}AQ=\mathrm{diag}(-i{\lambda }_j)$ andIn particular the determinant is nonzero, so $I+A$ is invertible. Applying the same argument to $-A$ proves that $I-A$ is also invertible. Equality holds precisely when all ${\lambda }_j=0$, equivalently $A=0$.

Let $A{v}_j=a_j{v}_j$. Commutation gives $A(B{v}_j)=B(A{v}_j)=a_{j}B{v}_j$, so $B$ preserves each eigenspace of $A$. Since the eigenvalues of $A$ are nondegenerate, each eigenspace is one-dimensional and $B{v}_j=b_j{v}_j$. The spectral theorem for a Hermitian matrix lets us choose these vectors as an orthonormal eigenbasis. With $U=(v_1,\dots ,v_n)$,This is simple spectrum gives simultaneous diagonalization. The common eigenvectors can be chosen up to their phases and ordering. In fact degeneracy does not prevent a common eigenbasis for commuting Hermitian matrices, but the one-dimensional argument already suffices here.

The convergent matrix exponential series respects similarity, so $e^A=U{e}^{{\Lambda }_A}{U}^{†}$ and likewise for $B$. Scalar exponentials multiply on each diagonal entry, givingThe essential hypothesis is commutation; Hermiticity supplies this convenient diagonalization proof.

The spectral theorem for a real symmetric matrix gives real eigenvalues and an orthonormal eigenbasis. Compute the roots of $\det (A-\lambda I)=0$, find a basis for each eigenspace, and orthonormalize within any repeated-eigenvalue eigenspace. Eigenvectors belonging to distinct eigenvalues are already orthogonal: if $Av=\lambda v$ and $Aw=\mu w$, symmetry gives $(\lambda -\mu )v^{T}w=0$.

Putting the normalized basis vectors in the columns of an orthogonal matrix $Q$ yieldsFor repeated eigenvalues, the choice of axes inside their eigenspace is not unique.

By the spectral theorem, a Hermitian matrix has real eigenvalues and an orthonormal eigenbasis. Choose column vectors $u_j$ with $H{u}_j={\lambda }_j{u}_j$ and $u_i^{†}{u}_j={\delta }_{ij}$, and form $U=(u_1\cdots u_n)$. Then $U^{†}U=I$, so $U$ is unitary, and $HU=U\Lambda$ givesThe entries of $\Lambda$ are the real eigenvalues, repeated according to their multiplicities. Within a degenerate eigenspace, any orthonormal basis may be used. Equivalently $H=U\Lambda U^{†}$.

We use the spectral theorem to construct a sufficiently accurate approximate eigenpair. Complete $x$ to an orthonormal eigenbasis $x,x_2,\dots ,x_n$ of the symmetric matrix $B$, with corresponding eigenvalues $\lambda ,{\lambda }_2,\dots ,{\lambda }_n$. Simplicity of $\lambda$ means that every ${\lambda }_j-\lambda$ below is nonzero. PutThe orthogonal decomposition of $Ex$ shows that $x^{T}v=0$ andWith $u_{\epsilon }=x+\epsilon v$ and ${\mu }_{\epsilon }=\lambda +\epsilon a$, direct expansion cancels every first-order term:Also $\Vert u_{\epsilon }{\Vert }_2^2=1+{\epsilon }^2\Vert v{\Vert }_2^2\ge 1$. Thus its eigenpair residual has Euclidean norm $O({\epsilon }^2)$ in Big O notation.

For completeness, a real symmetric matrix $M$ always has an eigenvalue within $\Vert (M-\mu I)u{\Vert }_2/\Vert u{\Vert }_2$ of any real $\mu$ and nonzero $u$. Indeed, writing $u={\sum }_j{c}_j{q}_j$ in an orthonormal eigenbasis with eigenvalues ${\theta }_j$ givesApply this symmetric eigenvalue residual bound to $M=B+\epsilon E$. There is an eigenvalue ${\lambda }_{\epsilon }$ such thatwhich provesThis establishes the first-order perturbation of a simple symmetric eigenvalue without presupposing differentiability of the perturbed eigenvector. If $n=1$, the sum defining $v$ is empty and the displayed eigenvalue formula is exact.

For $[a,b]=[1,2]$, $z(0)=-3$ and $\cosh \theta =3$ gives $e^{\theta }=3+2\sqrt{2}$. The normalization factors cancel, soIf the eigenvalues are known to be exactly $1,3/2,2$, use the more targeted annihilating polynomialIt equals one at zero and vanishes on the entire spectrum, so the spectral theorem gives $p_3(A)=0$. Thus $y^{(3)}=x$ in exact arithmetic. Degree three is the least degree that can vanish at all three distinct eigenvalues; a particular initial error supported on fewer eigenspaces can terminate earlier.

For a symmetric positive-definite matrix, the conjugate gradient method chooses the unique point in $x_0+K_k(A,r_0)$ minimizing the energy norm of the error. Equivalently it minimizes $\frac{1}{2}{y}^{T}Ay-b^{T}y$ on that affine space. Here $x_0=0$ and $r_0=b$.

Choose the eigenbasis orthonormally, as the spectral theorem permits, and write $x={\sum }_i{\alpha }_i{v}_i$. By part (a), every admissible error is $p(A)x={\sum }_i{\alpha }_{i}p({\lambda }_i)v_i$, $\mathrm{deg}p\le k$, $p(0)=1$. Orthogonality then givesThus the implicit CG polynomial minimizes exactly the displayed weighted sum. Strictly, the printed hypothesis that the eigenvectors are individually normalized must be supplemented by mutual orthogonality when eigenvalues repeat. Otherwise cross terms within a repeated-eigenvalue eigenspace need not vanish. An orthonormal eigenbasis always supplies the intended formulation.

If at most $\ell$ eigenvectors contribute to $x$, a polynomial with one factor $1-t/\lambda$ for each distinct contributing eigenvalue is admissible by step $\ell$ and makes that error zero. CG minimizes a nonnegative quantity, so $\boxed{\text{exact termination occurs in at most }\ell \text{ steps}}$.

For the last question, write $x=\alpha v_1+\beta v_2$ with eigenvalues $1,L$, $L=10^6$. The first polynomial has the form $p_1(t)=1-\eta t$. Differentiating its weighted objective givesIf the high-eigenvalue component is not exceptionally small, the $L$-weighted terms dominate and $\eta \approx 1/L$, so $p_1(1)\approx 1-10^{-6}\approx 1$: the first step mainly removes the high-eigenvalue error. This is the spectral weighting of conjugate gradient error. The expectation is conditional on the coefficients, not guaranteed by the two eigenvalues alone; if $\beta =0$, the first step instead solves the low-eigenvalue component exactly.

Put $B=A_1^T{A}_1$. It is a symmetric matrix and a positive semidefinite matrix, while $A_2^T{A}_2=I_n-B$ is also a positive semidefinite matrix. By the spectral theorem, choose an orthogonal matrix $V=[v_1,\dots ,v_n]$ such thatThe bounds follow directly from $z^{T}Bz=\Vert A_{1}z{\Vert }_2^2\in [0,\Vert z{\Vert }_2^2]$. Define the diagonal matricesThe columns of $A_{1}V$ are mutually orthogonal, since $(A_1{v}_i{)}^T(A_1{v}_j)={\lambda }_j{\delta }_{ij}$. For each ${\lambda }_j>0$, set $u_{1j}=A_1{v}_j/\sqrt{{\lambda }_j}$. These columns are orthonormal. At a zero ${\lambda }_j$, one has $A_1{v}_j=0$; choose $u_{1j}$ by completing the already prescribed columns to an orthonormal set of $n$ columns in ${\mathbb{R}}^k$. Such a completion exists because $k\ge n$. This gives $A_{1}V=U_1{\Sigma }_1$, including the zero columns.

Likewise, the Gram matrix of $A_{2}V$ is $\mathrm{diag}(1-{\lambda }_j)$. Normalize its nonzero columns and complete at its zero columns in ${\mathbb{R}}^{m-k}$, using $m-k\ge n$. The resulting $U_2$ has orthonormal columns and $A_{2}V=U_2{\Sigma }_2$. Combining the identities givesThis is a form of the cosine-sine decomposition. Completing the zero columns is essential: neither block was assumed to have full column rank.