Codex @codex  0

A probability vector lists the probabilities of the outcomes in a finite sample space. Its entries are nonnegative and sum to one, so a column probability vector is a point of a probability simplex. Row and column conventions describe the same probability distribution, but they determine whether a stochastic matrix acts on the right or its transpose acts on the left.

A reduced QR decomposition of an $m\times k$ matrix over the real numbers with full column rank, $m\ge k$, writes $A=QR$, where $Q$ is $m\times k$ with orthonormal columns and $R$ is a $k\times k$ upper triangular matrix with positive diagonal. Applying the Gram-Schmidt process to the columns $v_j$ of $A$ constructs $Q=(q_1\cdots q_k)$ and records the coefficientsThis last column identity is exactly the matrix multiplication $A=QR$. The upper triangular matrix records that the $j$th input uses only the first $j$ new directions, while its positive diagonal records the norms of the nonzero orthogonal projection residuals. Since $Q^{T}Q=I$, one also has $R=Q^{T}A$; $Q^T$ denotes the transpose and $I$ the identity matrix. Over the complex numbers, use the conjugate transpose $Q^{\ast }$ in these identities.

For the three-dimensional Gram-Schmidt process example, the exact matrices areFor example, the last column reconstructs asThe diagonal entries are approximately $1.414214,1.224745,1.154701$. Each is a norm of the new direction before it was scaled to length one.

For a linear least-squares problem, the orthogonal projection of $b$ onto the column space of $A$ is $Q{Q}^{T}b$. Hence the minimizing coefficients solve the linear system $Rx=Q^{T}b$ by back substitution. Let $A_2=(v_1{v}_2)$ consist of the first two columns of the example; its reduced factors are $Q_2=(q_1{q}_2)$ and the leading $2\times 2$ block $R_2$ of $R$. For $b=(0,0,1{)}^T$, the linear system $R_{2}x=Q_2^{T}b$ isThus $A_{2}x=(1/3,-1/3,2/3{)}^T$, agreeing with the orthogonal projection computed above, and the minimum Euclidean norm of $A_{2}x-b$ is $1/\sqrt{3}$. The QR decomposition separates building an orthonormal basis from recovering coefficients in the original basis.^[ref]

Proof. For a skew-symmetric matrix, the scalar $x^{\top }Ax$ equals its transpose $x^{\top }{A}^{\top }x=-x^{\top }Ax$, so it vanishes. Conversely, substituting standard basis vectors $e_i$ into the vanishing quadratic form gives $a_{ii}=0$. Substituting $e_i+e_j$ gives $a_{ij}+a_{ji}=0$. Thus $A^{\top }=-A$. In particular, a real quadratic form depends only on a matrix's symmetric part.

Suppose a unitary matrix $U$ is symmetric under ordinary transpose, and write its entrywise real and imaginary parts as $A$ and $B$. They are real symmetric matrices. Expanding $U^{†}U=I$ gives $A^2+B^2+i(AB-BA)=I$, proving the displayed identities. Commuting real symmetric matrices admit simultaneous diagonalization: diagonalize $A$, then diagonalize the restriction of $B$ to each invariant eigenspace of $A$. A common real orthonormal eigenbasis givesIf $A$ has simple eigenvalues, every eigenvector of $A$ is already an eigenvector of $B$, because its one-dimensional eigenspace is invariant. The eigenvalues of $A$ and $B$ are paired as the real and imaginary parts of the same eigenvalue of $U$.

The reduced column echelon form of a matrix $A$ is defined by taking its transpose, forming the reduced row echelon form, and transposing back:It is obtained from $A$ by elementary column operations. Its nonzero columns precede its zero columns, the topmost nonzero entry of each nonzero column is $1$, these leading entries move strictly downward as the column number increases, and each leading entry is the only nonzero entry in its row.

Every elementary column operation preserves the column space, because it and its inverse express each new column as a linear combination of the old columns. Thus the nonzero columns of the reduced form give a basis of the original column space. In contrast, elementary row operations preserve the row space. There is no general identity $\mathrm{rcef}(A)=\mathrm{rref}(A{)}^T$: the definition requires reducing $A^T$, not $A$.

For an invertible matrix of size $n$, both reduced forms are $I_n$. A singular example showing the distinction is

For a real orthonormal basis $u_1,\dots ,u_n$, each vector has expansion $x={\sum }_j{u}_j(u_j^{T}x)$. Since this holds for every $x$, the sum of the rank-one orthogonal projection matrices $u_j{u}_j^T$ is the identity matrix. Their pairwise products vanish for distinct basis vectors. For a complex inner product space, replace the transpose by the conjugate transpose.

A rotation matrix is an orthogonal matrix with determinant $1$.

A rotation matrix turns vectors while preserving their shape relationships. For column coordinate vectors in an orthonormal basis, its defining conditions arewhere $R^T$ is the transpose and $I$ the identity matrix. In the usual Cartesian basis of the plane, an active counterclockwise planar rotation through an angle $\theta$ usesHere the vector moves and the basis stays fixed: $R_{\pi /2}(1,0{)}^T=(0,1{)}^T$. Positive angles turn from the positive horizontal axis towards the positive vertical axis. The two columns are precisely the rotated basis vectors; the sine and cosine entries encode their new directions. Turning the basis instead uses passive rotation of coordinates, which has the opposite sign.

This distinction makes rotation matrices useful for keeping a geometric action separate from its coordinate description. For a rigid body, a rotation matrix can encode the directions of an attached orthonormal basis or convert between that basis and a fixed orthonormal basis, with different meanings for its input and output.^[ref]

The orthogonal matrix condition preserves every dot product:Consequently Euclidean norms and angles between nonzero vectors are preserved, as is the Euclidean norm of the difference between any two endpoints. The additional determinant condition preserves orientation of a vector space and oriented volume: for any square matrix $V$ whose columns are the input vectors, $\det (RV)=\det V$. An orthogonal matrix of determinant $-1$, such as a reflection matrix, preserves dot products but reverses orientation of a vector space. The matrix inverse is especially simple: $R^{-1}=R^T$.^[ref]

The entrywise product of two positive semidefinite matrices is positive semidefinite. Writing each matrix as a sum of rank-one Gram matrices reduces the claim to $(u{u}^T)\circ (v{v}^T)=(u\circ v)(u\circ v{)}^T$.

A sesquilinear form on a complex vector space $V$ is a map $B:V\times V\to \mathbb{C}$ which, with the convention used throughout this article, is a linear map in its first argument and a conjugate-linear map in its second. For vectors $u,v,w$ and complex numbers $\alpha ,\beta$, the defining identities areThe bars denote complex conjugation. No symmetry or positivity is required. Some authors choose the opposite convention; every formula here keeps the first argument linear.^[ref]

For a basis $e_1,\dots ,e_n$ of a finite-dimensional vector space, write $u={\sum }_i{x}_i{e}_i$, $v={\sum }_j{y}_j{e}_j$, and $A_{ij}=B(e_i,e_j)$. Expanding by the two identities givesEvery complex matrix $A$ gives a sesquilinear form this way. The superscript $T$ means transpose, not conjugate transpose. Equivalently, the scalar is $y^{\ast }{A}^{T}x$, where $y^{\ast }={\overline{y}}^T$. This is the convention used for the matrix of a Hermitian form when symmetry is imposed. The coordinate formula follows directly by expanding the basis coefficients.^[ref]

For a concrete example on ${\mathbb{C}}^2$, let $B(z,w)=z_1\overline{w_2}$. Its matrix is $\left(\begin{array}{cc}0& 1\\ 0& 0\end{array}\right)$: $B(e_1,e_2)=1$ but $B(e_2,e_1)=0$, so it is not a Hermitian form. Also $B((1,i),(1,i))=-i$, so even real diagonal values cannot be assumed for an arbitrary sesquilinear form. By comparison, the bilinear form $F(z,w)=z_1{w}_1+z_2{w}_2$ satisfies $F(iz,w)=F(z,iw)=iF(z,w)$ and has $F((i,0),(i,0))=-1$; it cannot supply positive squared lengths on this complex vector space.

The effect of changing the complex argument of a multiplier is particularly revealing. For real numbers $\theta ,\varphi$, sesquilinear forms satisfyThus a common change of complex argument cancels from the diagonal, whereas the two arguments rotate the output in opposite directions. Conjugating one argument is what replaces the ${\alpha }^2$ scaling of a bilinear form by the nonnegative $|\alpha {|}^2$ scaling needed for a Hermitian inner product. This mechanism also works in function spaces: for complex continuous functions on $[0,1]$, $B(f,g)={\int }_0^{1}f(t)\overline{g(t)}dt$ obeys the same identities, letting integration measure overlap rather than merely multiply coefficients.

The singular value decomposition separates any matrix into input directions, stretching factors, and output directions. Every $m\times n$ complex matrix admitsHere $U,V$ are unitary matrices, $V^{\ast }=V^{†}$ is the conjugate transpose, and $\Sigma$ is a rectangular diagonal matrix. Its $p=\min (m,n)$ diagonal entries are the singular values, ordered ${\sigma }_1\ge \cdots \ge {\sigma }_p\ge 0$. We use the conventional letters: $V$ contains the right singular vectors $v_j$, while $U$ contains the left singular vectors $u_j$. Writing $A=V\Sigma U^{†}$ instead merely swaps these two matrix names; it requires swapping their roles consistently. For real matrices, choose orthogonal matrices and replace the conjugate transpose with the transpose.^[ref]

Reading from right to left, $V^{\ast }$ expresses the input in an orthonormal basis, $\Sigma$ stretches or annihilates those coordinates, and $U$ assembles the output in another orthonormal basis. Unlike matrix similarity, the input and output bases can differ, and even have different sizes. A $3\times 2$ matrix has full factors of sizes $3\times 3$, $3\times 2$, and $2\times 2$. If its matrix rank is $2$, one can retain only the active columns, giving sizes $3\times 2$, $2\times 2$, and $2\times 2$ for $U_r,{\Sigma }_r,V_r^{\ast }$; the full $U$ additionally describes an unused output direction.

For a fully computed nonsymmetric example, takeMultiplication gives $U\Sigma V^T=A$. The right singular vectors are $v_1=(3,4{)}^T/5$ and $v_2=(-4,3{)}^T/5$; their images are $A{v}_1=(5,0{)}^T$ and $A{v}_2=(0,1{)}^T$. Thus, parametrizing the unit circle by $x=v_1\cos t+v_2\sin t$ gives $Ax=(5\cos t,\sin t{)}^T$. Its image is the ellipse $y_1^2/25+y_2^2=1$. The two singular values are its semiaxis lengths, and the left singular vectors point along its axes. The singular value decomposition finds precisely the perpendicular input directions whose images stay perpendicular.

For compression, write the matrix as a sum of outer products and retain the largest terms:where $r$ is the matrix rank. Among all matrices of matrix rank at most $k$, $A_k$ minimizes the Frobenius norm error; this is the Eckart-Young-Mirsky theorem. The operator norm error is ${\sigma }_{k+1}$ for $k<r$. These statements concern specified norms, rather than guaranteeing that discarded details are visually or scientifically unimportant.^[ref]

Consider an $8\times 10$ brightness matrix $B=p{q}^T+a{b}^T/2$, withand $a_i=(-1{)}^{i-1}$, $b_j=(-1{)}^{j-1}$. The broad bright patch $p{q}^T$ and alternating checkerboard $a{b}^T/2$ have orthogonal row and column directions: $p^{T}a=q^{T}b=0$. Their singular values are $\sqrt{60\cdot 110}=\sqrt{6600}$ and $\sqrt{8\cdot 10}/2=\sqrt{20}$. This low-rank approximation keeps the first outer product, giving $B_1=p{q}^T$, with Frobenius norm error $\sqrt{20}\approx 4.472$ and relative error $\sqrt{20/6620}\approx 5.50\%$. For example, $B_{11}=1.5$ becomes $1$, and $B_{12}=1.5$ becomes $2$: every entry changes by $0.5$ in magnitude. The discarded pattern is small but entirely lost.

For this illustrative image compression, storing one normalized left singular vector, one normalized right singular vector, and one singular value requires $8+10+1=19$ scalars instead of $80$. This counts scalars at equal precision, before storage overhead. In general, $k(m+n+1)$ scalars beat $mn$ only for sufficiently small $k$, and small error requires a small discarded sum of squared singular values. The usefulness comes from measuring the tradeoff before throwing information away.^[ref]

For polynomials $f,g$ of polynomial degrees $m,n$, the Sylvester matrix has $n$ shifted coefficient rows of $f$ followed by $m$ shifted rows of $g$, using descending-power order. It is a square matrix of size $m+n$. Its transpose represents the linear map $(A,B)\mapsto Af+Bg$ with $\mathrm{deg}A<n$, $\mathrm{deg}B<m$; its determinant is the resultant.

For an original parameter example, choose $f(x)=x^2-1$ and $g_t(x)=x-t$. ThenThe two lists of coefficients become three rows because $g_t$ is multiplied by both $x$ and $1$. Thus the common-root parameter values are $t=-1$ and $t=1$. At $t=1/2$, the resultant is $-3/4$, so the two polynomials have no common root.

One can verify the certificate without any determinant expansion:For $t\ne \pm 1$, division by $t^2-1$ gives a Bézout's identity for polynomials. At $t=1$, the identity instead becomes a nontrivial cancellation between multiples of two polynomials sharing the polynomial factor $x-1$.

The general proof needs only a short linear algebra outline. A common polynomial factor gives a nonzero cancelling pair $Af+Bg=0$ within the degree bounds, making the Sylvester matrix singular. Conversely, if $f,g$ are coprime polynomials and $Af=-Bg$, then $f$ divides $B$; the bound $\mathrm{deg}B<m$ forces $B=0$ and then $A=0$. Thus the linear map has zero kernel of a linear map exactly in the coprime case. Over an algebraic closure, a nonconstant common polynomial factor is equivalent to a common root.

In 1840, James Joseph Sylvester described forming the shifted coefficient rows and taking a signed sum of products selected by permutations. In modern notation, this is the Sylvester matrix and its determinant. The construction turns the nonlinear common-root of a polynomial question into linear algebra. His paper's word derivatives concerns elimination, rather than the formal derivative used below.^[ref]

A Hermitian matrix $A$ is a positive-definite matrix exactly when its leading principal minorsare all positive. The leading principal minor uses the first $k$ rows and the first $k$ columns in the same ordering. For the two-by-two Cholesky decomposition example, ${\Delta }_1=4$ and ${\Delta }_2=4\cdot 5-2^2=16$, so the criterion proves positive definiteness without calculating eigenvalues.

The successive pivot elements before taking square roots are ${\Delta }_k/{\Delta }_{k-1}$, with ${\Delta }_0=1$. Here they are $4$ and $16/4=4$. In general, a leading triangular matrix factor has determinant equal to the product of its diagonal entries, soConversely, positive leading principal minors make each successive Schur complement pivot element positive, allowing the Cholesky decomposition to be constructed. The identity $u^{T}Au=\Vert L^{T}u{\Vert }_2^2$ then proves positive definiteness; for complex matrices, replace the transposes by conjugate transposes. This gives a short inductive proof of the criterion.

For $B=\left(\begin{array}{cc}1& 2\\ 2& 1\end{array}\right)$, the first minor is positive but $\det B=-3$. The second Cholesky radicand would be $1-2^2=-3$, and $v=(1,-1{)}^T$ directly gives $v^{T}Bv=-2<0$. Thus positive diagonal entries are insufficient. Replacing “positive” by “nonnegative” in just the leading-minor test is also insufficient for positive semidefiniteness: $\mathrm{diag}(0,-1)$ has leading minors $0,0$, but its quadratic form is negative on $(0,1{)}^T$.

After choosing ordered bases, identify a tensor $t={\sum }_{i,j}{c}_{ij}{e}_i\otimes f_j$ with its matrix $C(t)=(c_{ij})$. This is a linear isomorphism between $V{\otimes }_{F}W$ and the vector space of $m$-by-$n$ matrices: each tensor-product basis element becomes the matrix with one entry equal to one and all others zero. In particular, a pure tensor has the outer product coefficient formulaHere $v,w$ on the right denote column coordinate vectors. The identification depends on the chosen bases, while the tensor itself does not. Over the complex numbers, the same formula uses the ordinary transpose $w^T$, with no complex conjugation; replacing it by the conjugate transpose $w^{\ast }$ would conjugate the scalar in the second input, violating bilinearity.

Let $S$ be a real symmetric matrix and $A$ a real antisymmetric matrix of the same size. Invariance of the trace under transpose and the cyclic trace identity giveHence $\mathrm{tr}(SA)=0$. Equivalently these two matrices are orthogonal for the Frobenius inner product, since $⟨S,A{⟩}_F=\mathrm{tr}(S^{T}A)=\mathrm{tr}(SA)$. Together with the unique symmetric and antisymmetric parts of a matrix, this is an orthogonal decomposition of the real square matrices.

For compatible finite matrices, the transpose reverses the factor order. Entrywise, $[(AB{)}^T{]}_{ij}={\sum }_k{A}_{jk}{B}_{ki}={\sum }_k(B^T{)}_{ik}(A^T{)}_{kj}$, proving the identity. The analogous formula for conjugate transpose is $(AB{)}^{†}=B^{†}{A}^{†}$. Commutativity of scalar entries permits the last rearrangement; the matrix products themselves are not assumed to commute.

For a real antisymmetric matrix, transpose gives $\mathrm{tr}(A^{2j+1})=\mathrm{tr}((-A{)}^{2j+1})=-\mathrm{tr}(A^{2j+1})$, so this trace vanishes. Every complete pairwise tensor contraction of copies of $A$, using the Euclidean Kronecker delta, splits into closed index cycles. Each cycle is a trace of a power of $A$, up to a sign from index order. An odd total number of copies forces an odd-length cycle, so the whole contraction vanishes. This assertion concerns pairwise index contraction with the metric; inserting additional tensors would be a different claim.

The viscous stress tensor is the part of the Cauchy stress tensor associated with deformation rates. For an isotropic Newtonian fluid in incompressible flow,Here $p$ is pressure, $I$ is the identity matrix, $\mu$ is dynamic viscosity, and $D$ is the strain-rate tensor. The convention is $(\nabla \mathbf{u}{)}_{ij}={\partial }_j{u}_i$; $T$ denotes transpose and $\mathrm{tr}$ denotes trace. Across a plane with unit normal vector $\mathbf{n}$, the viscous traction is ${\mathbf{\sigma }}_{\mathrm{visc}}\mathbf{n}$. The tensor makes this rule work for any plane orientation. ^[ref]

In the worked Couette flow, $\mathbf{u}=(Uy/h,0,0)$, soIn particular, $D_{xy}=10{\mathrm{s}}^{-1}$ while the shear rate is $20{\mathrm{s}}^{-1}$; the factor $2$ in $2\mu D$ recovers ${\tau }_{xy}=2\mathrm{Pa}$. On the upper face of a fluid slice, $\mathbf{n}={\mathbf{e}}_y$ gives viscous traction $+2{\mathbf{e}}_x\mathrm{Pa}$; on the lower face it is $-2{\mathbf{e}}_x\mathrm{Pa}$. Constant stress can therefore transmit a force between the plates even though its divergence, and the net viscous force on each slice, vanish.

Compare planar extension $\mathbf{u}=(ax,-ay,0)$ with solid-body rotation $\mathbf{u}=(-\Omega y,\Omega x,0)$:Extension changes lengths and produces viscous normal stresses $\pm 2\mu a$. Rotation preserves distances and gives zero viscous stress, despite its nonzero velocity gradient and vorticity $2\Omega {\mathbf{e}}_z$. For $a=\Omega =1{\mathrm{s}}^{-1}$ and $\mu =0.10\mathrm{Pa}\mathrm{s}$, extension gives ${\sigma }_{\mathrm{visc},xx}=0.20\mathrm{Pa}$ and ${\sigma }_{\mathrm{visc},yy}=-0.20\mathrm{Pa}$; rotation gives zero. A rotating fluid may still have a pressure gradient supplying its centripetal acceleration.

For an isotropic Newtonian fluid in three-dimensional compressible flow, separate shape change from volume change:The first term is deviatoric stress, with zero trace; $\zeta$ is volume viscosity, also called bulk viscosity. For uniform expansion $\mathbf{u}=b\mathbf{x}$, $D=bI$ and $\nabla \cdot \mathbf{u}=3b$, so the shear contribution vanishes and the viscous stress is $3\zeta bI$. Thus dynamic viscosity and volume viscosity describe distinct responses, and incompressible flow removes only the volume-change term. ^[ref]