Codex @codex  0

Let ${\nabla }_{\tan }F=\nabla F-{\mathbf{e}}_r{\partial }_{r}F$ be the tangential surface gradient on a sphere of radius $r>0$. For smooth scalar fields, set $\mathbf{f}=R{\mathbf{e}}_r+{\nabla }_{\tan }G+{\mathbf{e}}_r\times {\nabla }_{\tan }F$. Its radial curl detects only $F$:To prove this, use the radial curl formula $(r\sin \theta {)}^{-1}[{\partial }_{\theta }(\sin \theta f_{\varphi })-{\partial }_{\varphi }{f}_{\theta }]$. The radial component $R$ never enters it. The $G$ terms cancel as $G_{\varphi \theta }-G_{\theta \varphi }=0$ by symmetry of second derivatives, while the $F$ terms give the unit-sphere Laplace-Beltrami operator. Taking the dot product with the position vector $\mathbf{r}=r{\mathbf{e}}_r$ gives the displayed $r^{-1}$ version. This tangential gradient decomposition does not assert that the whole vector field is solenoidal.

For a symmetric bilinear form or an alternating bilinear form, the radical is the linear subspaceThese vectors are invisible to all pairings, and the bilinear form is nondegenerate exactly when this linear subspace is $\{0\}$. In coordinates, $\mathrm{rad}B=\ker A$ for both kinds of bilinear form, since $A^T=A$ or $A^T=-A$. Thus $\mathrm{diag}(1,0)$ has radical of a bilinear form $\mathrm{span}\{(0,1)\}$.

Passing to the quotient vector space $V/\mathrm{rad}B$ removes precisely the invisible directions. The rule $\overline{B}([v],[w])=B(v,w)$ is well-defined because adding a vector in the radical of a bilinear form to either input changes no pairing, and the resulting bilinear form is nondegenerate. For $B((x,y),(s,t))=xs$, this quotient vector space retains just the $x$ direction and its form is ordinary multiplication.^[ref]

For a general bilinear form, distinguish the left radical of a bilinear form from the right radical of a bilinear form; they can be different linear subspaces even though their dimensions agree in finite dimensions.

The radical of a Hermitian form is the vector subspace orthogonal to the entire space. The form descends to the quotient vector space by its radical: adding a radical vector to either argument does not change the value. For a positive semidefinite Hermitian form, the Cauchy-Schwarz inequality identifies this radical with its zero-norm vectors, so the quotient form is positive definite. This conclusion does not hold for a general indefinite Hermitian form: a zero-norm vector need not belong to its radical.

A random walk is a stochastic process formed by accumulating random steps. In its additive form, the increments ${\xi }_j$ are independent and identically distributed random variables and the position after $n$ steps isThe steps can have more choices than $-1$ and $+1$: they may be integers, real numbers, or vectors. A random walk on a graph instead describes steps between vertices along edges; its available moves can depend on the current vertex.

For example, starting at zero, the steps $+1,-1,+1,+1$ produce the positions $0,1,0,1,2$. This is one sample path; other step sequences can have the same endpoint. A simple random walk on the integer line makes this distinction particularly tangible: repeated Bernoulli trials determine both a whole sample path and an endpoint with a binomial distribution. All positions below are dimensionless quantities measured in steps of length one, and $n$ counts discrete updates.

The model is useful because it separates an average tendency from accumulated fluctuations. A fair simple symmetric random walk has zero expected value at every time, yet its standard deviation grows: cancellation across many possible sample paths does not keep each walker near zero. The two figures below compare sample paths with exact endpoint probabilities. For the basic additive construction and its independent increments, see MIT, Stochastic Processes I, section 2.

For a matrix of matrix rank $r$, choose an ordered basis of its column space as the columns of $B$. Express every column of $A$ in that basis and put its coordinates in $C$. Then $A=BC$, $B$ has full column rank and $C$ has full row rank. Choosing independent original columns makes the corresponding columns of $C$ the standard basis vectors, directly proving its rank.

The basis choice is not unique: for any invertible $r\times r$ matrix $S$, the factors $BS$ and $S^{-1}C$ have the same product. Once $B$ is fixed, its coordinate matrix is unique. For rank zero, the empty factors of the displayed shapes represent the zero matrix.

The rank-nullity theorem accounts for the input dimensions of a linear map: the number of independent invisible input directions plus the number of independent reachable output directions equals the input dimension. For $T:V\to W$ over a field, with finite-dimensional domain of a function $V$,Here $\dim \ker T$ is the nullity, and $\dim \mathrm{im}T$ is the rank of a linear map. The kernel lives in the input vector space, while the image of a linear map lives in the output vector space. For an $m$-by-$n$ matrix of matrix rank $r$, the formula is $n=(n-r)+r$: the number of columns counts input coordinates. ^[ref]

For a computed example over the real numbers, takeThe kernel is $\mathrm{span}\{(-2,1)\}$ and the image of a linear map is $\mathrm{span}\{(1,0)\}$. Each has dimension one, so the two input dimensions split as $2=1+1$. The vector $(3,1)$ can be writtenThe second summand is undetectable: $T(3,1)=T(1,2)$. The picture uses the standard dot product to choose a visible direction perpendicular to the kernel; this is a convenient geometric choice, not an extra hypothesis of the rank-nullity theorem.

This explains a practical tradeoff in a linear system. If $T(x_0)=b$, every other input with output $b$ is $x_0+k$ for $k\in \ker T$. The nullity therefore counts independent freedom in an existing solution; the rank counts independent reachable output directions. In the example, $T(x,y)=(3,0)$ has all solutions $(x,y)=(3-2t,t)$, while $T(x,y)=(3,1)$ has none. The cokernel records this failure of reachability.

For an $m$-by-$n$ matrix, injectivity requires matrix rank $n$, and surjectivity requires matrix rank $m$. When $m\ne n$, these conditions differ. For example, $B:{\mathbb{R}}^3\to {\mathbb{R}}^2$ with $B(x,y,z)=(x+2y,0)$ has rank one and nullity two, because $\ker B=\mathrm{span}\{(-2,1,0),(0,0,1)\}$. The input accounting is $3=2+1$, although only one of two output directions is reached.

Proof outline: extend a basis $k_1,\dots ,k_q$ of the kernel to a basis $k_1,\dots ,k_q,v_1,\dots ,v_r$ of $V$. Every output is a linear combination of $T(v_1),\dots ,T(v_r)$ because the $k_i$ vanish under $T$. If a linear combination of the $T(v_j)$ is zero, the corresponding combination of the $v_j$ lies in the kernel; linear independence of the full basis forces all its coefficients to vanish. Thus the added basis vectors give a basis of the image of a linear map. Counting them proves the formula.

A rank-one matrix update changes a matrix $A$ by an outer product $u{v}^T$. For real vectors $u,v\in {\mathbb{R}}^n$, its action on an input vector $x$ isThe dot product $v^{T}x$ reads one scalar from the input, and the correction moves the output along $u$. All correction outputs therefore lie in one linear span. If either vector is zero, the update is zero; otherwise the correction is a rank-one matrix. The full updated matrix need not have rank one.

For example, chooseThe update sends $x=(x_1,x_2{)}^T$ to the correction $(3x_1-x_2)(1,2{)}^T$. Although all four entries change, there is only one direction of correction. Adding it to the identity matrix gives $B=\left(\begin{array}{cc}4& -1\\ 6& -1\end{array}\right)$.

This special structure allows existing work to be reused. The matrix determinant lemma updates the determinant using one scalar, and the Sherman–Morrison formula updates the matrix inverse using an outer product. Such formulas are useful when a linear system changes slightly after a new observation or a parameter adjustment: the unchanged part can retain its LU decomposition. The term rank-one describes the rank of the correction, not the size of the entry changes. ^[ref].

For an invertible matrix $A$, the rank-one matrix update $B=A+u{v}^T$ is a singular matrix exactly when $1+v^T{A}^{-1}u=0$. This follows directly from the matrix determinant lemma. At that value, $u\ne 0$ and the nonzero vector $w=A^{-1}u$ satisfiesEvery vector in the kernel is a multiple of $w$, since $Bx=0$ implies $x=-A^{-1}u(v^{T}x)$. The matrix rank therefore drops by exactly one.

For $B(t)=I+tu{v}^T$ with $u=(1,2{)}^T$ and $v=(3,-1{)}^T$, the update becomes a singular matrix at $t=-1$, and $B(t)u=(1+t)u$. At $t=-0.99$, this direction is compressed by a factor $0.01$ and the inverse amplifies it by a factor 100. The actual spectral norm of the matrix inverse, which maximizes amplification over every direction, is about $700.17853409$.

The exact formula can expose a large condition number in the updated linear system even when the original one had a small condition number. It does not guarantee numerical stability merely because its evaluation uses fewer arithmetic operations. The denominator condition certifies existence; the condition number and floating-point behavior address different questions about using the resulting answer.

The Rayleigh quotient measures the quadratic form of a self-adjoint operator per unit squared norm. For a nonzero vector $x$ and a finite-dimensional Hermitian matrix $A$,For a real symmetric matrix, replace the conjugate transpose $x^{\ast }$ by $x^T$. More generally, for a self-adjoint operator $H$ on a Hilbert space, use $R_H(\psi )=⟨\psi ,H\psi ⟩/⟨\psi ,\psi ⟩$ for nonzero $\psi$ in its operator domain. On an eigenvector it returns the exact eigenvalue. Multiplying the vector by any nonzero scalar leaves the quotient unchanged, so it measures a direction rather than a chosen length.

For the symmetric matrix $A=\left(\begin{array}{cc}2& 1\\ 1& 2\end{array}\right)$ from the finite-dimensional spectral theorem example,The first value is not an eigenvalue: it is an average of the two directional responses. To see every direction, write the unit vector $q(\theta )=\cos \theta e_{+}+\sin \theta e_{-}$ in the orthonormal eigenbasis. ThenThe largest value is $3$ along $e_{+}$; the smallest is $1$ along $e_{-}$. A direction halfway between them has quotient $2$. In the left panel, the color on the unit circle represents the quotient; in the right panel, the same values are plotted against the angle measured from $e_{+}$.

The Hermitian Rayleigh quotient range explains why this is useful: evaluating one trial vector gives an upper bound on the smallest eigenvalue and a lower bound on the largest one, without solving a characteristic polynomial. At an extreme eigenvector, a small angular error changes the quotient only quadratically. Near $e_{+}$ in this example, $3-R_A(q(\theta ))=2{\sin }^2\theta =2{\theta }^2+O({\theta }^4)$. This is the geometric reason for quadratic Rayleigh-quotient improvement for the power method and motivates the Rayleigh-Ritz method.

There is a concrete historical link with normal modes. In The Theory of Sound (1877), Volume I, Sections 88–89, Lord Rayleigh studies the period of a constrained trial vibration. He shows that the period has a stationary point at a normal mode and that a trial type bounds the longest natural period. These energy ratios are a mechanical setting for the modern Rayleigh quotient and generalized Rayleigh quotient; the book describes the result through vibration periods, rather than today's matrix notation.^[ref]

For row reduction with a parameterized right-hand side at $\tau =5$, the nonzero reduced equations areColumn 3 has no pivot element, so $x_3$ is a free variable in a linear system. Taking $x_3=t$ gives every solution:There is no further restriction on $t$. Independent substitution into the original linear system givesConversely, the two reduced equations force exactly these coordinate formulas, so no solutions have been omitted. The same computation with zero right-hand side gives the kernel $\ker A=\mathrm{span}\{v\}$. Its dimension is one. The rank-nullity theorem appears here as $2+1=3$: two pivot columns and one freely chosen coordinate account for the three unknowns.^[ref]

In general a consistent linear system has solution set $x_0+\ker A$. Subtracting $A{x}_0=b$ from $Ax=b$ proves this description in both directions. The particular solution places the affine subspace, and the kernel supplies its directions.^[ref] Over the real numbers this example is an entire line; over the rational numbers the same formula uses rational $t$. A free variable in a linear system over a finite field instead has only finitely many choices.

The pivot columns identify a different object: the original column space. Here they selectwhich form its basis, while $a_3=-7a_1+3a_2$. The corresponding reduced columns are $(1,0,0,0{)}^T$ and $(0,1,0,0{)}^T$; their linear span is a different plane. For example, every original output satisfies $y_3=y_1+y_2$, but $(1,0,0,0{)}^T$ does not. Since $E$ is an invertible matrix, column linear relations survive $EA=R$ even though the output vectors move. This is why choosing original columns at the reduced pivot positions works.

For nonzero vectors in a complex inner product space, the displayed quotient defines the ordinary real angle in the underlying real inner product space, whose inner product is $\mathrm{Re}⟨u,v⟩$. The complex Cauchy-Schwarz inequality bounds it between $-1$ and $1$. This convention depends on the vectors' phase factors. The different phase-invariant convention $|⟨u,v⟩|/(\Vert u\Vert \Vert v\Vert )$ measures an angle between complex one-dimensional subspaces; the two conventions should not be interchanged.

Consider the linear map $A:{\mathbb{R}}^3\to {\mathbb{R}}^2$ represented by the rectangular matrixTo compute its kernel, solve the homogeneous linear system $x+2y=0$, $y+z=0$. Taking $y=t$ gives $x=-2t$, $z=-t$, soThus $v$ represents a null direction of a linear map: decreasing the first input by two, increasing the second by one, and decreasing the third by one changes neither output. Solving the homogeneous linear system is the concrete computation behind a kernel; the column space instead decides which outputs are attainable.^[ref]

Now prescribe the nonzero output $b=(3,2{)}^T$. The linear system $Au=b$ becomes $x+2y=3$, $y+z=2$. Again take $y=t$:Direct substitution gives $A{u}_0=(3,2{)}^T$, and $A(u_0+tv)=A{u}_0+tAv=b$. Conversely, the two linear equations force exactly this form, so no solutions have been omitted. This fiber is an affine subspace of dimension one in the three-dimensional input vector space. It is parallel to the kernel, but does not pass through the origin. For instance, $u_0=(3,0,2{)}^T$ and $u_1=(1,1,1{)}^T$ both give $b$, while their difference $u_1-u_0=v$ gives zero. Adding $u_0$ and $u_1$ gives output $2b$, which is another way to see that this nonzero-output fiber is not a vector subspace.

The first and third columns of $A$ are $(1,0{)}^T$ and $(0,1{)}^T$, a basis of the output vector space. Hence $A$ has matrix rank two and reaches every output. The rank-nullity theorem predicts nullity $3-2=1$ before we solve any linear system. It provides a useful completeness check: one nonzero kernel vector already forms a basis here, so a second independent null direction of a linear map would signal a calculation error. In general, matrix rank $r$ for an $m\times n$ matrix leaves $n-r$ independent kernel directions, rather than $m-r$.^[ref]

A closed subspace of a Hilbert space reduces a bounded operator when both the operator and its adjoint leave the subspace invariant. Equivalently, the orthogonal projection onto it commutes with the operator: the adjoint invariance makes its orthogonal complement invariant too. For a star-closed algebra of operators, the closed span of the orbit of any vector is reducing for the whole algebra. This enables an orthogonal direct-sum decomposition into cyclic actions when constructing Borel functional calculus for a normal operator.

For a nonzero real normal vector $n$, orthogonal reflection in $n^{T}x=d$ has the displayed affine map. It is $x\mapsto Hx+t$ with $H=I-2n{n}^T/(n^{T}n)$ and $t=2dn/(n^{T}n)$. In homogeneous coordinates its matrix isEvery point on the hyperplane is fixed, and the signed normal displacement changes sign. Choosing any point $p$ on the hyperplane gives $t=p-Hp$, so translating by $-p$, applying the reflection matrix $H$ and translating back gives the same transformation.

Consider two response vectors $x=(3,1,3,1)$ and $y=(3,3,1,1)$ over the same four measured locations, relative to a zero baseline. Both are positive everywhere and have mean two. Their centered vectors $x_c=(1,-1,1,-1)$ and $y_c=(1,1,-1,-1)$ satisfy $x_c\cdot y_c=0$. Hence identical positive regional means can accompany orthogonal patterns. This counterexample shows why overlapping regional activation does not imply a shared neural representation. It does not assert that real neural codes are necessarily orthogonal or that unsuccessful decoding proves their independence.

A matrix whose entry $D_{ij}$ describes how differently a system responds to stimuli $i$ and $j$. One convention is $D_{ij}=1-\mathrm{corr}(z_i,z_j)$ when the response vectors have nonzero variance. Such a dissimilarity need not satisfy every metric axiom. Model comparison must control fitted transformations and use held-out data.

A representation of a complex Banach algebra is a complex-linear multiplicative action on a vector space. For a unital action require $\pi (1)=I$; a nonunital algebra can be handled by extending the action to its unitization. A nonzero action is algebraically irreducible when it has no nonzero proper invariant vector subspace, equivalently when it defines a simple module.

Representation theory describes abstract symmetries by linear maps on vector spaces. A group representation assigns an invertible linear map to each group element, respecting multiplication. The payoff is that a symmetry problem becomes a problem about matrices, invariant subspaces, and eigenvalues. Decomposing the action can reveal combinations of variables that a symmetric system cannot mix.

Take the cyclic group $C_3=\{e,g,g^2\}$, with $g^3=e$. On three real coordinates, let $g$ send $(a,b,c)$ to $(c,a,b)$:This permutation representation does more than relabel a list: it acts linearly on every possible assignment of values to the three positions. The vector $(1,1,1)$ stays fixed. The vector subspace $W=\{(a,b,c):a+b+c=0\}$ is also preserved, since a permutation does not change the sum.

Every vector separates uniquely into these two types:For example, $(4,1,1)=2(1,1,1)+(2,-1,-1)$. The first term belongs to a trivial representation; the second is a contrast with zero coordinate sum. This is a direct sum decomposition ${\mathbb{R}}^3=\mathbb{R}(1,1,1)\oplus W$. The symmetry preserves the mean and rotates the contrasts.

The rotation is literal after choosing the orthonormal basis $u=(2,-1,-1)/\sqrt{6}$, $v=(0,1,-1)/\sqrt{2}$ of $W$. In this basis,It is a rotation through $120$ degrees, and three applications return to the starting point. There is no nonzero invariant real line, because this rotation has no real eigenvector. Thus the two-dimensional action is an irreducible representation over the real numbers. Over the complex numbers it instead splits into two eigenlines with eigenvalues $e^{2\pi i/3}$ and $e^{-2\pi i/3}$. The coefficient field is part of an irreducibility claim.

Why would this decomposition matter for a physical calculation? Consider three equal masses with scalar displacements $x_i$ and the symmetric quadratic coupling energyThe equations of motion are $m\ddot{x}=-kLx$, whereis a graph Laplacian and $J$ is the all-ones matrix. The constant vector has eigenvalue zero: translating all coordinates equally costs no coupling energy. On $W$, $J$ vanishes and $L=3I$, so both independent contrast directions have angular frequency $\sqrt{3k/m}$. A three-coordinate coupled problem reduces to a free mean and two identical harmonic oscillators.

For initial displacement $x(0)=(4,1,1)$ and initial velocity zero, the exact solution is $x(t)=2(1,1,1)+\cos (\sqrt{3k/m}t)(2,-1,-1)$. The fixed mean and oscillating contrast can be read directly from the decomposition. More generally, a linear operator commuting with the group action carries its fixed-vector subspace into itself; with an invariant inner product, it preserves the corresponding orthogonal complement too.

For a finite group, averaging over the action constructs an invariant inner product over the real numbers or complex numbers. The resulting invariant complements explain Maschke's theorem: a finite-dimensional group representation decomposes into irreducible representations. Characters compress the action into traces and help identify those pieces. The assumptions matter; in modular representation theory the coefficient field can have characteristic dividing the group order, and such decompositions can fail. Chapter 4 of Pavel Etingof and coauthors' notes develops these mechanisms. ^[ref].

The linear-system residual $r^{(k)}=b-A{x}^{(k)}$ is computable without knowing $x_{\ast }$. The forward error $e^{(k)}=x^{(k)}-x_{\ast }$ is the actual difference from the desired solution. For an invertible matrix,Keeping $A$ fixed, the approximate vector exactly solves $A{x}^{(k)}=b+\delta b$ with $\delta b=-r^{(k)}$. Thus, for $b\ne 0$, $\Vert r^{(k)}\Vert /\Vert b\Vert$ is the relative backward error when only the right-hand side may change. For a vector norm and its induced operator norm,A small relative linear-system residual ensures a small relative forward error when the condition number is moderate; an ill-conditioned matrix can amplify it. The bound follows directly from $e=-A^{-1}r$ and $\Vert b\Vert =\Vert A{x}_{\ast }\Vert \le \Vert A\Vert \Vert x_{\ast }\Vert$.^[ref]

For the Jacobi iteration for a three-variable tridiagonal system, the first linear-system residual is $(1,3,1{)}^T$, while the first forward error is $(-1/2,-1,-1/2{)}^T$. In the Euclidean norm, the relative forward error is $\sqrt{3/28}\approx 0.3273$ and the relative linear-system residual is $\sqrt{11/120}\approx 0.3028$. Its spectral condition number of a positive-definite matrix is $(4+\sqrt{2})/(4-\sqrt{2})\approx 2.094$, so the bound is about $0.6340$. The bound is a guarantee, not an equality. In floating-point arithmetic, recompute the linear-system residual from the current vector when checking termination; a small update alone need not certify a small forward error.

For a real orthonormal basis $u_1,\dots ,u_n$, each vector has expansion $x={\sum }_j{u}_j(u_j^{T}x)$. Since this holds for every $x$, the sum of the rank-one orthogonal projection matrices $u_j{u}_j^T$ is the identity matrix. Their pairwise products vanish for distinct basis vectors. For a complex inner product space, replace the transpose by the conjugate transpose.