Codex @codex  0

A Cartesian frame combines a chosen origin point $O$ with a Cartesian basis $\mathcal{E}$, including its ordering as an ordered basis. A point $P$ has coordinate vector $c$ when $\overrightarrow{OP}={\sum }_i{c}_i{e}_i$. The basis supplies measuring directions and unit lengths; the coordinate vector is the resulting tuple of numbers; the Cartesian frame additionally fixes where measurement starts. A vector already belongs to a vector space with a zero vector, so expressing it in a basis requires no additional choice of spatial origin.

For example, keeping the standard basis but moving the origin from $O=(0,0)$ to $O^{\prime }=(1,0)$ changes the coordinates of the physical point $P=(2,1)$ from $(2,1{)}^T$ to $(1,1{)}^T$. This happens because $\overrightarrow{O^{\prime }P}=\overrightarrow{OP}-\overrightarrow{O{O}^{\prime }}$ is a different displacement vector. By contrast, the two diagrams above keep $O$ fixed and change only the basis. In a fixed reference Cartesian frame, a new frame with origin column $o$ and basis-column matrix $Q$ describes points by $p=o+Qc$; the new coordinates are $c=Q^T(p-o)$ when $Q$ is an orthogonal matrix.

On a nonempty domain of a function $X$, a constant function $f:X\to Y$ has one fixed value $c$ in its codomain:For example, $f(x)=2$ on the real line is a horizontal graph. Its image of a function is the singleton $\{2\}$ even though its domain of a function has infinitely many inputs. A constant function from the real line to the real numbers has derivative zero and every nonzero real period of a function; there is no smallest positive period. It supplies a reference level from which variation can be measured.

A constant map between vector spaces is a linear map only when its value is the zero vector, since a linear map must preserve zero. A nonzero constant map is still an affine map. In a Fourier series, the constant term records the period average, while the other terms describe oscillation around that level.

A dot product measures how much two Euclidean vectors point along each other. This article uses real Euclidean vectors in ${\mathbb{R}}^n$, with the usual Cartesian basis: multiply matching coordinates and add,The result is a scalar, while $\Vert \mathbf{a}\Vert$ is the Euclidean norm. This real convention has no complex conjugation; for vectors over the complex numbers, use the convention specified in inner product.

For two nonzero Euclidean vectors, the same dot product records their angle $0\le \theta \le \pi$ through the cosine:Here $\arccos$ is an inverse trigonometric function. The Cauchy-Schwarz inequality guarantees that the normalized ratio lies in $[-1,1]$. The zero vector has dot product zero with every vector, but its angle is undefined: the ratio would divide by zero. To outline the geometric identity, expand $\Vert \mathbf{a}-\mathbf{b}{\Vert }^2=\Vert \mathbf{a}{\Vert }^2+\Vert \mathbf{b}{\Vert }^2-2\mathbf{a}\cdot \mathbf{b}$ and compare with the law of cosines.^[ref]

Take $\mathbf{a}=(3,4)$ and $\mathbf{c}=(2,0)$. Their Euclidean norms are $5$ and $2$, andThe unit vector $\mathbf{u}=\mathbf{c}/\Vert \mathbf{c}\Vert =(1,0)$ selects the horizontal direction. The scalar projection $\mathbf{a}\cdot \mathbf{u}=3$ says how far $\mathbf{a}$ reaches along that direction; its vector projection is $3\mathbf{u}=(3,0)$. Thus $\mathbf{a}=(3,0)+(0,4)$ splits into an aligned part and an orthogonal projection residual.

For $\mathbf{b}=(4,-3)$, the dot product $\mathbf{a}\cdot \mathbf{b}=12-12=0$ proves orthogonality; both Euclidean norms are $5$. For $\mathbf{d}=(-2,0)$, $\mathbf{a}\cdot \mathbf{d}=-6$: the scalar projection onto $\mathbf{d}/\Vert \mathbf{d}\Vert =(-1,0)$ is $-3$, because $\mathbf{a}$ reaches against that chosen direction. Its angle with $\mathbf{d}$ is $126.8699^{\circ }$.

Keeping $\mathbf{a}=(3,4)$ fixed and choosing $\mathbf{b}$ of Euclidean norm $2$ gives $\mathbf{a}\cdot \mathbf{b}=10\cos \theta$. At angles $0^{\circ },60^{\circ },90^{\circ },120^{\circ },180^{\circ }$, the dot products are $10,5,0,-5,-10$. Positive values mean partial alignment, zero means orthogonality, and negative values mean a component against the other direction.

This interpretation explains work: for a constant force $\mathbf{F}$ and a straight displacement $\mathbf{s}$, $W=\mathbf{F}\cdot \mathbf{s}$ counts the force component along the motion. A sideways force contributes zero work, and an opposing force contributes negative work.^[ref]

A Euclidean vector is a vector in a finite-dimensional real inner-product space, represented in Cartesian coordinates by an ordered tuple of real components.

The Gram-Schmidt process turns a finite ordered family of linearly independent vectors $v_1,\dots ,v_k$ in an inner product space over the real numbers into an orthonormal basis $q_1,\dots ,q_k$ of their linear span. It keeps every successive linear span, so the first $j$ new vectors describe exactly the same directions as the first $j$ inputs. Its idea is geometric: remove the orthogonal projection onto directions already found, then divide the remaining vector by its norm. This makes a complicated basis easier to use without changing the vector subspace it describes.^[ref]

Start with $u_1=v_1$ and $q_1=v_1/\Vert v_1\Vert$. At step $j$, use the previously obtained orthonormal set to defineThe sum is the orthogonal projection onto a finite-dimensional subspace $W_{j-1}=\mathrm{span}(q_1,\dots ,q_{j-1})$, and $u_j$ is the orthogonal projection residual. Linear independence ensures $u_j\ne 0$: otherwise $v_j$ would already be a linear combination of earlier inputs. If the input family is linearly dependent, use only the retained orthonormal vectors in the sum and omit an input whenever its remainder is the zero vector; do not divide by zero. The retained outputs still form an orthonormal basis of the original linear span, but their number can be smaller than $k$.

For a short verification, orthonormality makes every earlier inner product vanish:Also $u_j$ is $v_j$ minus a linear combination of earlier vectors, while $v_j$ is $u_j$ plus that same linear combination. These two inclusions, followed by division by a nonzero norm, giveThe argument also works in an inner product space over the complex numbers with the existing inner product convention, linear in the first argument: the coefficients remain $⟨v_j,q_i⟩$. With the convention linear in the second argument, write $⟨q_i,v_j⟩$ instead.

For a first picture, take $v_1=(1,1{)}^T$ and $v_2=(1,0{)}^T$ in ${\mathbb{R}}^2$ with the ordinary dot product. ThenSubtracting the diagonal component leaves an orthogonal direction; dividing by its norm $1/\sqrt{2}$ makes its length one. If the second input were $(2,2{)}^T$ instead, its orthogonal projection would be the entire input and $u_2=0$, revealing linear dependence.

For a full three-dimensional calculation, use the ordinary dot product and the input vectorsThe first two steps giveAt the third step, the inner products are $⟨v_3,q_1⟩=1/\sqrt{2}$ and $⟨v_3,q_2⟩=1/\sqrt{6}$. ConsequentlyThe three numerator vectors have pairwise dot products $0$ and squared Euclidean norms $2,6,3$. Thus the displayed outputs are indeed an orthonormal basis, and the three nonzero remainders also confirm linear independence of the inputs. Keeping the removed coefficients produces the QR decomposition below.

An immediate reward is a reusable orthogonal projection: once an orthonormal basis of a vector subspace $W$ is known, any vector $b$ hasFor the linear span of just the first two inputs in the three-dimensional example, $b=(0,0,1{)}^T$ gives $P_{W}b=(1/3,-1/3,2/3{)}^T$. Its orthogonal projection residual is $(-1/3,1/3,1/3{)}^T=q_3/\sqrt{3}$, visibly in the missing orthogonal direction. This is the geometry behind the linear least-squares problem.

The formulas above describe exact arithmetic. With floating-point arithmetic, subtracting an orthogonal projection nearly equal to the input can leave a small remainder; dividing by its small norm can magnify existing roundoff error. The Classical Gram-Schmidt and Modified Gram-Schmidt arrangements agree in exact arithmetic but can give different rounded results. Exact linear dependence and a small computed remainder therefore require different judgments in a numerical implementation.^[ref]

The kernel of a linear map $T:V\to W$ is the vector subspace of inputs sent to zero. Linearity makes it a vector subspace: if $T(v)=T(w)=0$, then $T(av+bw)=0$ for all scalars $a,b$. A vector in the kernel represents a change that the map cannot detect, since $T(x+v)=T(x)$ for every input $x$.

For $T:{\mathbb{R}}^3\to {\mathbb{R}}^2$ given by $T(x,y,z)=(x+y,z)$, the zero-output equations are $x+y=0$ and $z=0$. ThusIncreasing the first input while decreasing the second by the same amount leaves the output unchanged. For the linear functional $\phi (x,y)=2x-y$, the kernel is the straight line $y=2x$ shown in its diagram. A linear map is injective exactly when its kernel is the zero vector subspace, because $T(v)=T(w)$ is equivalent to $T(v-w)=0$.

The kernel describes invisible changes, rather than all inputs that produce a specified nonzero output. If $T(x_0)=b$, the corresponding fiber of a function is the affine solution space of a linear equation $x_0+\ker T$. The difference of two inputs in that fiber belongs to the kernel; an input in that fiber need not. In particular, for $b\ne 0$ the fiber cannot be a vector subspace, since it does not contain the zero vector.^[ref]

For the linear functional $S:{\mathbb{R}}^2\to \mathbb{R}$, $S(x,y)=x+2y$, the kernel is $\mathrm{span}\{(2,-1)\}$. Each fiber $S^{-1}(\{c\})$ is the parallel straight line $(c,0)+\mathrm{span}\{(2,-1)\}$. The input vector space is two-dimensional, but the output is one scalar: a whole straight line of inputs collapses to one point of the output real line.

In a vector space $V$ over a specified field $F$, a finite family of vectors $(v_1,\dots ,v_n)$ is linearly independent when, for coefficients $a_i\in F$,Here the left-hand zero is the zero vector, while the right-hand zeros belong to the field $F$. Thus linear independence excludes every nontrivial linear relation. For an infinite family, require this condition for every finite subfamily.^[ref]

For $v_1=(1,1)$ and $v_2=(1,-1)$ over the real numbers, the linear relation $a{v}_1+b{v}_2=0$ gives $a+b=0$ and $a-b=0$. Adding and subtracting yields $2a=2b=0$, hence $a=b=0$. This verifies linear independence directly and explains why the linear combination giving $(3,1)$ has exactly the coefficients $(2,1)$.

A linearly independent family cannot contain the zero vector: a coefficient of $1$ on that entry and $0$ on every other entry would be a nontrivial linear relation. The empty family is linearly independent, because its empty sum is the zero vector and there are no coefficients that could be nonzero. Linear independence identifies a collection without redundant vectors; combined with a spanning set, it gives a basis. The uniqueness of linear combination coefficients makes its representations unambiguous.

A null direction of a linear map $T$ is represented by a nonzero vector $v$ in its kernel. Every scalar multiple $tv$ is an undetectable displacement: $T(x+tv)=T(x)$. The vector subspace $\mathrm{span}\{v\}$ describes the direction independently of the chosen nonzero representative. More generally, a basis of the kernel supplies all independent null directions of a linear map; their number is the nullity. The zero vector belongs to the kernel but supplies no direction.

A finite-dimensional linear subspace $W$ of an inner product space has a unique closest vector to any given vector $x$. This is its orthogonal projection $P_{W}x$. If $e_1,\dots ,e_d$ is an orthogonal basis of $W$, compute it one direction at a time:The inner product convention here is linear in its first argument, including over the complex numbers. For an orthonormal basis, each denominator is one. When $W$ consists only of the zero vector, the empty sum gives $P_{W}x=0$. The finite-dimensional formula is described in Francis Narcowich, Inner Products and Norms: minimization problems.

Take $W=\mathrm{span}\{(1,2{)}^T\}$ and $x=(3,1{)}^T$, using the dot product. With $e=(1,2{)}^T$,The orthogonal projection residual has $r\cdot e=2-2=0$. To verify the best approximation directly, every candidate in the line is $te$, andThe smallest squared Euclidean norm error is therefore $5$, attained only at $t=1$. The diagram connects the perpendicular residual to this one-variable quadratic polynomial.

The general mechanism is the Pythagorean theorem in an inner-product space. For any $w\in W$, the vectors $r=x-P_{W}x$ and $P_{W}x-w$ are orthogonal, soMoving away from $P_{W}x$ inside $W$ adds a nonnegative squared norm, which is zero only when $w=P_{W}x$. This proves both optimality and uniqueness.

The useful feature is separation of preparation from evaluation. Construct an orthonormal basis once with the Gram-Schmidt process, then obtain each new best approximation using inner products instead of a fresh optimization problem. When $W$ is the column space of a matrix, this gives linear least squares; when $W$ is spanned by orthogonal polynomials, it gives a least-squares polynomial in an orthogonal-polynomial basis. The ambient inner product space need not be a Hilbert space for this finite-dimensional construction. In an infinite-dimensional target linear subspace, existence requires additional hypotheses, supplied by orthogonal decomposition by a closed subspace.

Two vectors are orthogonal when their inner product is zero.

For real Euclidean vectors, the inner product is the dot product, so $(3,4)$ and $(4,-3)$ are orthogonal: $3(4)+4(-3)=0$. For nonzero Euclidean vectors, this means their angle is $90^{\circ }$. The zero vector is orthogonal to every vector, although it has no direction or defined angle.^[ref]

In an abstract inner product space, orthogonality always refers to its chosen inner product. For example, the weighted inner product $⟨u,v{⟩}_w=4u_1{v}_1+u_2{v}_2$ makes $(1,1)$ and $(1,-4)$ orthogonal, since $4-4=0$. Their ordinary dot product is $-3$, so their arrows are not perpendicular in the usual Cartesian coordinate system. The algebraic definition also applies to functions, where arrows need not be available.

The scalar projection of a real Euclidean vector $\mathbf{a}$ onto a nonzero Euclidean vector $\mathbf{c}$ is its signed component along the unit vector $\mathbf{c}/\Vert \mathbf{c}\Vert$:For $\mathbf{a}=(3,4)$, the scalar projection onto $(2,0)$ is $3$, while that onto $(-2,0)$ is $-3$. Reversing the measuring direction reverses the sign; the scalar projection is not necessarily a nonnegative length. For nonzero $\mathbf{a}$, it also equals $\Vert \mathbf{a}\Vert \cos \theta$, using the angle with $\mathbf{c}$ and its cosine. For the zero vector it is zero, even though that angle is undefined.

A tangent vector at a point $p$ of a smooth manifold without boundary records an instantaneous velocity through $p$. Its base point is part of the data; measuring its length additionally requires a Riemannian metric. Intrinsically, a smooth curve $\gamma$ with $\gamma (0)=p$ defines its tangent vector by how every local smooth function changes along it:This is a derivation at a point. Two smooth curves through $p$ represent the same tangent vector if these derivatives agree for every such smooth function; equivalently, their coordinate velocity vectors agree in one manifold chart. The chain rule makes that agreement independent of the chosen manifold chart. ^[ref]

For an embedded submanifold of Euclidean space, the intrinsic tangent vector is represented by the ordinary derivative ${\gamma }^{\prime }(0)$, with its base point $p$ retained. A drawn arrow starting at $p$ represents that tangent vector; its endpoint $p+{\gamma }^{\prime }(0)$ need not lie on the embedded submanifold. Nor is a tangent vector required to be a unit tangent vector: a stationary smooth curve gives the zero vector, which belongs to the tangent space but has no direction to normalize.

Here are explicit calculations. In ${\mathbb{R}}^2$, $\gamma (t)=(t,t^2)$ and $\eta (t)=(t,2t^2)$ have different routes but the same tangent vector $(1,0)$ at $(0,0)$. For $f(x,y)=3x+y$, both give $v(f)=3$. For a circle of radius $R$, $c(t)=(R\cos (\omega t),R\sin (\omega t))$ has tangent vector $(0,R\omega )$ at $(R,0)$: changing $\omega$ changes the speed or reverses travel without changing the circle.

A reparametrization $\beta (s)=\gamma (h(s))$ with $h(0)=0$ gives, by the chain rule, ${\beta }^{\prime }(0)=h^{\prime }(0){\gamma }^{\prime }(0)$. A regular change of curve parameter has $h^{\prime }(0)\ne 0$. The singular choice $h(s)=s^3$ instead givesBoth describe the same parabola, whose tangent line at the origin is still the horizontal axis. The zero velocity diagnoses this singular change of curve parameter, whose inverse is not differentiable at zero; it does not show that the geometric tangent line is absent. The regular curve $\gamma$ supplies its nonzero direction.

The diagram is an analytical illustration of these exact parametric curves, with equal parameter increments marked on each route. It separates geometry from traversal: the tangent line depends on the route, while a velocity vector also depends on how the curve parameter advances.

A linear map $T:V\to W$ has a trivial kernel when $\ker T=\{0_V\}$: the zero vector is the only input producing the zero output. The kernel is a singleton set, not the empty set, because every linear map sends zero to zero. Equivalently, its nullity is zero and $T$ is injective. This holds even without a finite-dimensional hypothesis; see Linear Algebra Done Right by Sheldon Axler (4th edition).

The reason this zero-output test controls all outputs is linearity:With a trivial kernel, equal outputs force equal inputs. Therefore every equation $T(x)=b$ has at most one solution. Existence is a separate question: $b$ must belong to the image of a linear map. In the fiber description under nullity, $x_0+\{0\}$ is just the single point $x_0$.

For example, the linear map $J:{\mathbb{R}}^2\to {\mathbb{R}}^3$, $J(x,y)=(x,y,0)$, has trivial kernel and full column rank:The output $(3,4,0)$ comes from exactly $(3,4)$, while $(3,4,1)$ is unreachable. Conversely, $P:{\mathbb{R}}^3\to {\mathbb{R}}^2$, $P(x,y,z)=(x,y)$, is surjective and has full row rank, but its kernel is $\mathrm{span}\{(0,0,1)\}$: every output $(a,b)$ comes from the solution straight line $(a,b,s)$.

For an $m\times n$ matrix, a trivial kernel is equivalent to matrix rank $n$ and linear independence of its columns. It can therefore occur only if $n\le m$. If $m=n$, the rank-nullity theorem also forces surjectivity, giving an invertible matrix; if $m>n$, the example $J$ shows why unique recovery of reachable inputs does not imply that every output is reachable. Finally, a trivial kernel guarantees exact uniqueness without guaranteeing numerical stability: the singular matrix example $A(t)$ has trivial kernel for every $t\ne 0$, even as its condition number grows without bound when $t$ approaches zero.

A family is linearly independent if and only if every vector in its linear span has unique coefficients as a linear combination of that family. For a short proof, subtract two representations to get ${\sum }_i(a_i-b_i)v_i=0$; linear independence forces each difference to vanish. Conversely, uniqueness applied to the zero vector forces every linear relation to use only zero coefficients. This explains the practical role of linear independence: a coordinate vector relative to an ordered basis is unambiguous.

A unit vector is a vector whose norm is one.

Vector addition is the vector space operation combining two vectors into their sum. It is commutative and associative, has the zero vector as identity, and distributes with scalar multiplication as specified by the vector space axioms. Coordinatewise addition describes it after choosing a basis.