Inner product ($⟨\cdot ,\cdot ⟩$)

An inner product assigns a scalar $⟨u,v⟩$ to two vectors and supplies their lengths and orthogonality. Using the convention linear in the first argument, it satisfiesThe resulting norm is $\Vert v\Vert =\sqrt{⟨v,v⟩}$. Over the real numbers, conjugation has no effect and the form is symmetric and bilinear. Over the complex numbers, it is sesquilinear; the convention linear in the second argument is also common, so formulas for coefficients must match the stated convention.

The Euclidean example is $⟨u,v⟩={\sum }_i{u}_i\overline{v_i}$. For real $u=(2,1)$ and $v=(1,3)$ this gives $⟨u,v⟩=5$, while $\Vert u{\Vert }^2=5$ and $\Vert v{\Vert }^2=10$. If $r=(-1,2)$, then $⟨r,u⟩=0$: perpendicularity becomes an algebraic equation. The Cauchy-Schwarz inequality gives $|⟨u,v⟩|\le \Vert u\Vert \Vert v\Vert$, and in a real inner product space it makes the angle formula $\cos \theta =⟨u,v⟩/(\Vert u\Vert \Vert v\Vert )$ meaningful for nonzero vectors.

Different inner products express different measurements. The weighted example $⟨u,v⟩=4u_1{v}_1+u_2{v}_2$ on ${\mathbb{R}}^2$ gives $\Vert (x,y){\Vert }^2=4x^2+y^2$, so its unit-length vectors lie on an ellipse. For functions, an integral such as $⟨f,g⟩=\int f\overline{g}$ supplies the geometry of the L2 space. This lets orthogonal projection and least-squares solutions apply to curves as well as coordinate lists.

At distinct real nodes $x_0,\dots ,x_n$ with positive weights $w_j$, the displayed expression is an inner product on the real vector space of polynomials of degree at most $n$. Symmetry and bilinearity follow from finite sums. If its squared norm vanishes, positivity of every weight forces $p(x_j)=0$ at every node. The root bound for a nonzero polynomial then forces $p=0$, proving positive definiteness.

More generally, on any vector subspace of real functions, the evaluation form is an inner product exactly when the evaluation map $p\mapsto (\sqrt{w_j}p(x_j){)}_j$ is injective. On polynomials of degree at most $n+1$ it fails: the nonzero node polynomial ${\prod }_j(x-x_j)$ vanishes at all nodes. This gives a positive semidefinite form with a nontrivial kernel. On the smaller polynomial space, the Lagrange interpolation polynomial identifies evaluation coordinates with ${\mathbb{R}}^{n+1}$, and Cauchy-Schwarz inequality follows either from the Euclidean inequality in those coordinates or from minimizing $\Vert p-tq{\Vert }^2$ over real $t$.

On real symmetric matrices of size $n\ge 2$, the displayed pairing is symmetric because $\det (AC)=\det A\det C$. It is not bilinear: $B(I,2I)=2^n$, whereas linearity would require $2B(I,I)=2$. Its self-pairing is always nonnegative, but a nonzero singular matrix has $B(A,A)=0$, so positive definiteness fails as well. For $n=1$ it reduces to the ordinary real inner product. The Frobenius inner product $\mathrm{tr}(A^{T}C)$ supplies the appropriate bilinear, positive-definite geometry in every size.

Let $P_n(\mathbb{R})$ be the vector space of real polynomials of degree of a polynomial at most $n$. Choose $n+1$ distinct real sample points $t_0,\dots ,t_n$ and positive real weights $w_0,\dots ,w_n$. Thenis an inner product. Symmetry and bilinearity follow from linearity of polynomial evaluation. Its value at $(f,f)$ is nonnegative, and equality forces all $f(t_j)=0$. By the root bound for a polynomial, a nonzero polynomial of degree of a polynomial at most $n$ cannot vanish at $n+1$ distinct points. Thus the form has positive definiteness.

Equivalently, the polynomial evaluation map on a finite point set identifies $P_n(\mathbb{R})$ with ${\mathbb{R}}^{n+1}$, equipped with a weighted Euclidean inner product. Too few sample points destroy positive definiteness: if the distinct points number at most $n$, the nonzero polynomial ${\prod }_j(x-t_j)$ lies in $P_n(\mathbb{R})$ and has zero sampled squared norm.

For three equally weighted samples at $-1,0,1$, an orthonormal basis isThis construction gives a finite sampling version of orthogonal polynomials and permits exact Gram-Schmidt orthogonalization using only finitely many evaluations.

An inner product space is a real or complex vector space equipped with a positive-definite inner product. Over the real numbers the form is symmetric and bilinear; over the complex numbers it is conjugate symmetric and sesquilinear. It defines the norm $\Vert v\Vert =\sqrt{⟨v,v⟩}$. The Cauchy-Schwarz inequality proves the triangle inequality for this norm. A complete inner product space is a Hilbert space; every finite-dimensional inner product space is complete.

Averaging squared norms of all signed sums of finitely many vectors in an inner-product space cancels every mixed term. This equates the average to the sum of the individual squared norms, distinguishing Hilbert-space geometry from other sequence-space norms.

The Fischer inner product on polynomials is $⟨x^{\alpha },x^{\beta }{⟩}_F=\alpha !{\delta }_{\alpha \beta }$. Multiplication by $x_i$ is adjoint to differentiation by $x_i$. Therefore multiplication by $|x{|}^2$ is adjoint to the ordinary Laplacian, yielding the harmonic decomposition of homogeneous polynomials by finite-dimensional orthogonality.

For complex matrices, the inner product $⟨A,B{⟩}_F=\mathrm{tr}(A^{\ast }B)$ is conjugate-linear in its first argument. On real symmetric matrices it becomes $\mathrm{tr}(AB)$. In particular $⟨A,x{x}^T{⟩}_F=x^{T}Ax$, identifying quadratic forms with linear tests on matrices.

For the real two-by-two matrix space, the Frobenius inner product is $\mathrm{tr}(A^{t}B)={\sum }_{i,j}{a}_{ij}{b}_{ij}$. The displayed four matrices form an orthonormal basis. The first spans the identity direction, the next two span the traceless symmetric matrices, and the last spans the antisymmetric matrices. Symmetric and antisymmetric matrices are orthogonal because transposition changes the sign of their trace pairing; pairing with the identity gives the trace. This basis therefore displays the orthogonal direct sum into these three natural pieces.

Let $S$ be a real symmetric matrix and $A$ a real antisymmetric matrix of the same size. Invariance of the trace under transpose and the cyclic trace identity giveHence $\mathrm{tr}(SA)=0$. Equivalently these two matrices are orthogonal for the Frobenius inner product, since $⟨S,A{⟩}_F=\mathrm{tr}(S^{T}A)=\mathrm{tr}(SA)$. Together with the unique symmetric and antisymmetric parts of a matrix, this is an orthogonal decomposition of the real square matrices.

The discrete L2 inner product is the cell-volume-weighted Hermitian inner product on a uniform grid. Its associated norm is the discrete L2 norm. It provides the finite-dimensional analogue of spatial integration by parts and continuous energy method identities.

For a positive weight $w$, the weighted inner product of functions is $⟨f,g{⟩}_w=\int \overline{f}gw$. It is the usual inner product in the L2 space for the measure $w(x)dx$. A Sturm-Liouville eigenfunction expansion uses this weight to make its eigenfunctions orthogonal.

The weight specifies how different locations contribute to a squared error: $\Vert f-g{\Vert }_w^2=\int |f-g{|}^{2}w$. More weight makes an error in that region more costly. The weighted Euclidean inner product makes the same choice for a finite list of coordinates.

For an explicit real-function example on $[0,1]$, let $w(x)=1+x$ and approximate $f(x)=x$ by a constant function. The orthogonal projection onto a finite-dimensional subspace spanned by the function $1$ givesThe best unweighted constant is $1/2$. Giving greater weight to larger $x$ moves the best constant upward, so a weighting choice changes the approximation even though the allowed family of functions is unchanged.

A positive measurable weight $w$ defines the displayed norm on the L2 space for the measure $w(x)dx$, with functions identified when equal almost everywhere. This is the norm induced by the weighted inner product. Changing $w$ changes the contribution of each region to approximation error. If $p_0,\dots ,p_N$ are nonzero mutually orthogonal real functions, completing squares shows that the unique best approximation to real $f$ in their linear span has coefficients $a_j=⟨f,p_j{⟩}_w/\Vert p_j{\Vert }_w^2$.

A symmetric positive-definite matrix $M$ defines an inner product on real coordinate vectors byFor diagonal $M$, positive diagonal entries weight the squared coordinate errors. This is the finite-coordinate version of a weighted inner product; the positive-definite matrix example appears in Francis Narcowich, Inner Products and Norms, standard inner products, example 8.

For $M=\mathrm{diag}(4,1)$, the squared norm of $(a,b{)}^T$ is $4a^2+b^2$. A first-coordinate error of one costs as much as a second-coordinate error of two. The unit sphere is the ellipse $4a^2+b^2=1$, and orthogonality means $4u_1{v}_1+u_2{v}_2=0$. Thus “closest” and “perpendicular” depend on the chosen inner product. The orthogonal projection residual example shows how this changes a best approximation.

A dot product measures how much two Euclidean vectors point along each other. This article uses real Euclidean vectors in ${\mathbb{R}}^n$, with the usual Cartesian basis: multiply matching coordinates and add,The result is a scalar, while $\Vert \mathbf{a}\Vert$ is the Euclidean norm. This real convention has no complex conjugation; for vectors over the complex numbers, use the convention specified in inner product.

For two nonzero Euclidean vectors, the same dot product records their angle $0\le \theta \le \pi$ through the cosine:Here $\arccos$ is an inverse trigonometric function. The Cauchy-Schwarz inequality guarantees that the normalized ratio lies in $[-1,1]$. The zero vector has dot product zero with every vector, but its angle is undefined: the ratio would divide by zero. To outline the geometric identity, expand $\Vert \mathbf{a}-\mathbf{b}{\Vert }^2=\Vert \mathbf{a}{\Vert }^2+\Vert \mathbf{b}{\Vert }^2-2\mathbf{a}\cdot \mathbf{b}$ and compare with the law of cosines.^[ref]

Take $\mathbf{a}=(3,4)$ and $\mathbf{c}=(2,0)$. Their Euclidean norms are $5$ and $2$, andThe unit vector $\mathbf{u}=\mathbf{c}/\Vert \mathbf{c}\Vert =(1,0)$ selects the horizontal direction. The scalar projection $\mathbf{a}\cdot \mathbf{u}=3$ says how far $\mathbf{a}$ reaches along that direction; its vector projection is $3\mathbf{u}=(3,0)$. Thus $\mathbf{a}=(3,0)+(0,4)$ splits into an aligned part and an orthogonal projection residual.

For $\mathbf{b}=(4,-3)$, the dot product $\mathbf{a}\cdot \mathbf{b}=12-12=0$ proves orthogonality; both Euclidean norms are $5$. For $\mathbf{d}=(-2,0)$, $\mathbf{a}\cdot \mathbf{d}=-6$: the scalar projection onto $\mathbf{d}/\Vert \mathbf{d}\Vert =(-1,0)$ is $-3$, because $\mathbf{a}$ reaches against that chosen direction. Its angle with $\mathbf{d}$ is $126.8699^{\circ }$.

Keeping $\mathbf{a}=(3,4)$ fixed and choosing $\mathbf{b}$ of Euclidean norm $2$ gives $\mathbf{a}\cdot \mathbf{b}=10\cos \theta$. At angles $0^{\circ },60^{\circ },90^{\circ },120^{\circ },180^{\circ }$, the dot products are $10,5,0,-5,-10$. Positive values mean partial alignment, zero means orthogonality, and negative values mean a component against the other direction.

This interpretation explains work: for a constant force $\mathbf{F}$ and a straight displacement $\mathbf{s}$, $W=\mathbf{F}\cdot \mathbf{s}$ counts the force component along the motion. A sideways force contributes zero work, and an opposing force contributes negative work.^[ref]

For two sparse vectors stored as increasing nonzero-coordinate streams, advance whichever current index is smaller; that coordinate has no possible partner in the other remaining stream. At equal indices, accumulate the product and advance both. The loop invariant accounts for all matched coordinates already passed and excludes any future match for skipped smaller coordinates. Termination occurs when either stream ends. The method uses $O(k_1+k_2)$ time and constant auxiliary space, with floating-point results subject to the chosen evaluation order.

For nonzero real vectors in a Euclidean space, their angle $\theta \in [0,\pi ]$ is determined by the displayed dot product formula. The Cauchy-Schwarz inequality puts the quotient in $[-1,1]$, so the inverse cosine is defined. Perpendicular vectors have angle $\pi /2$, parallel vectors pointing the same way have angle zero, and those pointing in opposite directions have angle $\pi$. A zero vector has no uniquely defined direction and hence no angle by this definition.

For nonzero vectors in a complex inner product space, the displayed quotient defines the ordinary real angle in the underlying real inner product space, whose inner product is $\mathrm{Re}⟨u,v⟩$. The complex Cauchy-Schwarz inequality bounds it between $-1$ and $1$. This convention depends on the vectors' phase factors. The different phase-invariant convention $|⟨u,v⟩|/(\Vert u\Vert \Vert v\Vert )$ measures an angle between complex one-dimensional subspaces; the two conventions should not be interchanged.

The scalar projection of a real Euclidean vector $\mathbf{a}$ onto a nonzero Euclidean vector $\mathbf{c}$ is its signed component along the unit vector $\mathbf{c}/\Vert \mathbf{c}\Vert$:For $\mathbf{a}=(3,4)$, the scalar projection onto $(2,0)$ is $3$, while that onto $(-2,0)$ is $-3$. Reversing the measuring direction reverses the sign; the scalar projection is not necessarily a nonnegative length. For nonzero $\mathbf{a}$, it also equals $\Vert \mathbf{a}\Vert \cos \theta$, using the angle with $\mathbf{c}$ and its cosine. For the zero vector it is zero, even though that angle is undefined.

The vector projection multiplies the scalar projection by the chosen unit vector, giving the orthogonal projection onto the linear span of a nonzero real Euclidean vector $\mathbf{c}$:For $\mathbf{a}=(3,4)$ and $\mathbf{c}=(2,0)$, this gives $\mathbf{p}=(6/4)(2,0)=(3,0)$ and $\mathbf{r}=(0,4)$; the dot product $\mathbf{r}\cdot \mathbf{c}=0$ checks the orthogonal projection residual. Replacing $\mathbf{c}$ by $(-2,0)$ gives the same $\mathbf{p}=(-6/4)(-2,0)=(3,0)$: reversing direction changes the scalar projection, but preserves the vector projection onto the line.

For a short proof of the nearest-point property, any vector $\mathbf{z}$ on this line satisfies $\Vert \mathbf{a}-\mathbf{z}{\Vert }^2=\Vert \mathbf{r}{\Vert }^2+\Vert \mathbf{p}-\mathbf{z}{\Vert }^2$ by the Pythagorean identity. The second term is uniquely minimized at $\mathbf{z}=\mathbf{p}$. This connects a directional measurement to the orthogonal projection used in approximation.^[ref]

For $x,y\in {\mathbb{F}}_2^n$, the binary inner product is $x\cdot y={\sum }_i{x}_i{y}_i$ in ${\mathbb{F}}_2$. Its value is the parity of the coordinates at which both vectors are one.

An orthonormal set consists of vectors that have norm one and are pairwise orthogonal. Its Gram matrix is the identity matrix.

A real $m\times k$ matrix whose columns are an orthonormal set, equivalently $Q^{T}Q=I_k$. In the complex case use the conjugate transpose instead. This requires $k\le m$; only in the square case does the transpose also provide a two-sided inverse, making $Q$ an orthogonal matrix.

An orthonormal basis of a Hilbert space is an orthonormal set whose linear span has the whole space as its closure. This completeness of an orthonormal system allows every vector to be expanded in the basis vectors, with convergence in norm. In finite dimension it is also a basis in the finite-linear-combination sense.

An orthonormal set is complete in a Hilbert space when the closure of its linear span is the whole space. Equivalently, the only vector orthogonal to every member is zero: project any vector onto the closed span, and its remaining orthogonal component must vanish. This is completeness of an expansion system, distinct from completeness of the ambient metric space.

Coordinates relative to an origin $\mathbf{o}$ and an orthonormal basis ${\mathbf{e}}_1,\dots ,{\mathbf{e}}_n$ preserve Euclidean distances: if $\mathbf{x}-\mathbf{y}={\sum }_i{d}_i{\mathbf{e}}_i$, then $\Vert \mathbf{x}-\mathbf{y}{\Vert }^2={\sum }_i{d}_i^2$. This matters when finding the eccentricity of a conic section in an oblique plane: arbitrary projected coordinates need not preserve the ratio of its semi-major axis to its semi-minor axis.

A complete orthonormal basis in a Hilbert space resolves the identity as ${\sum }_j|j⟩⟨j|=I$. In infinite dimension the partial sums converge strongly: their action on each fixed vector converges in norm. Generalized position and momentum eigenstates instead give distributional integral resolutions, such as $\int dx|x⟩⟨x|=I$, with their normalization interpreted through the Dirac delta function.

For a real orthonormal basis $u_1,\dots ,u_n$, each vector has expansion $x={\sum }_j{u}_j(u_j^{T}x)$. Since this holds for every $x$, the sum of the rank-one orthogonal projection matrices $u_j{u}_j^T$ is the identity matrix. Their pairwise products vanish for distinct basis vectors. For a complex inner product space, replace the transpose by the conjugate transpose.

For vectors $v_1,\dots ,v_n$, the Gram matrix isTwo finite vector families with the same Gram matrix are related by an isometry between their spans; in finite dimensions that isometry can be extended to a unitary map after completing orthonormal bases.

A real matrix product giving pairwise dot products of columns, for $T^{\top }T$, or rows, for $T{T}^{\top }$. It is a positive semidefinite matrix. The kernel of a Gram product is determined by the corresponding factor, which makes these products useful for converting a zero quadratic form into an exact null space condition.

For real vectors $x_i$ in a Euclidean inner product space, their Gram matrix $G_{ij}=x_i^T{x}_j$ satisfies the displayed identity by expanding the squared norm. Thus every Gram matrix is a positive semidefinite matrix, and it is a positive-definite matrix exactly when the vector family is linearly independent. The complex version replaces transposes by conjugate transposes and uses $c^{\ast }Gc=\Vert {\sum }_i{c}_i{x}_i{\Vert }^2$.

For $n\ge 2$, the Gram matrix of $n$ unit vectors with common distinct-pair inner product $t$ is $G=(1-t)I+t11^T$. Its eigenvalues are $1+(n-1)t$ on the all-ones direction and $1-t$ on its orthogonal complement. It is a positive-definite symmetric matrix exactly when $-1/(n-1)<t<1$, permitting a linearly independent realization in ${\mathbb{R}}^n$. An explicit construction takes the columns ofThe two orthogonal projections have product zero, so $B^{T}B=G$; positive eigenvalues also make $B$ invertible. At either endpoint the corresponding eigenspace becomes a linear dependence.

The determinant of a Gram matrix is nonnegative and vanishes precisely when its vectors are linearly dependent. In Euclidean three-dimensional space, three column vectors form a square matrix $V$, whose Gram matrix is $V^{T}V$. Its determinant is $(\det V{)}^2=[v_1\cdot (v_2\times v_3){]}^2$, the square of the scalar triple product and of the parallelepiped volume.

For a basis $(u_i)$ of a finite-dimensional inner product space with conjugate-linear first argument, define $G_{ij}=(u_i,u_j)$ and ${\hat{u}}_k={\sum }_l(G^{-1}{)}_{lk}{u}_l$. Then $(u_i,{\hat{u}}_k)={\delta }_{ik}$ and $({\hat{u}}_j,{\hat{u}}_k)=(G^{-1}{)}_{jk}$. This is the inner-product representation of the dual basis; requiring the dual vectors to lie in the given span is essential.

For a matrix $G$ with independent standard normal entries, $G^{T}G/n$ is a Gaussian empirical Gram matrix and has expectation equal to the identity. With mean known to be zero it is the raw empirical second moment. Subtracting a sample mean would give a different estimator. On any fixed unit vector its quadratic form is a chi-squared variable divided by $n$.

For $G$ with $n$ independent standard Gaussian rows in dimension $k$, combine a constant-radius sphere metric net, the quadratic form net bound, the chi-squared concentration inequality, and a union bound. This gives $P(\Vert G^{T}G/n-I_k{\Vert }_{\mathrm{op}}>1/2)\le 2\exp (k\log B-n/1024)$ for a numerical $B$. Taking $n$ larger than a constant times $k\log p$ gives exponential decay in $k\log p$. The subspace is fixed, so no union over supports is needed.

If $E=A^{T}A/d-I$ has entrywise maximum norm at most $t$, every vector $z$ supported on at most $s$ coordinates satisfies $|z^{T}Ez|\le t\Vert z{\Vert }_1^2\le st\Vert z{\Vert }_2^2$. Thus the squared norm distortion is at most $st$ on all such sparse vectors. Use the quadratic inequality at $z=0$, where a ratio would be undefined.

If $u$ and $v$ are orthogonal, then

A finite-dimensional linear subspace $W$ of an inner product space has a unique closest vector to any given vector $x$. This is its orthogonal projection $P_{W}x$. If $e_1,\dots ,e_d$ is an orthogonal basis of $W$, compute it one direction at a time:The inner product convention here is linear in its first argument, including over the complex numbers. For an orthonormal basis, each denominator is one. When $W$ consists only of the zero vector, the empty sum gives $P_{W}x=0$. The finite-dimensional formula is described in Francis Narcowich, Inner Products and Norms: minimization problems.

Take $W=\mathrm{span}\{(1,2{)}^T\}$ and $x=(3,1{)}^T$, using the dot product. With $e=(1,2{)}^T$,The orthogonal projection residual has $r\cdot e=2-2=0$. To verify the best approximation directly, every candidate in the line is $te$, andThe smallest squared Euclidean norm error is therefore $5$, attained only at $t=1$. The diagram connects the perpendicular residual to this one-variable quadratic polynomial.

The general mechanism is the Pythagorean theorem in an inner-product space. For any $w\in W$, the vectors $r=x-P_{W}x$ and $P_{W}x-w$ are orthogonal, soMoving away from $P_{W}x$ inside $W$ adds a nonnegative squared norm, which is zero only when $w=P_{W}x$. This proves both optimality and uniqueness.

The useful feature is separation of preparation from evaluation. Construct an orthonormal basis once with the Gram-Schmidt process, then obtain each new best approximation using inner products instead of a fresh optimization problem. When $W$ is the column space of a matrix, this gives linear least squares; when $W$ is spanned by orthogonal polynomials, it gives a least-squares polynomial in an orthogonal-polynomial basis. The ambient inner product space need not be a Hilbert space for this finite-dimensional construction. In an infinite-dimensional target linear subspace, existence requires additional hypotheses, supplied by orthogonal decomposition by a closed subspace.

In the weighted inner product $⟨f,g⟩={\int }_0^{1}xf(x)g(x)dx$, the best approximation to $x^2$ in the span of $1,x$ is the displayed polynomial. The normal equations are $1/4-A/2-B/3=0$ and $1/5-A/3-B/4=0$. Their solution gives $A=-3/10$, $B=6/5$. The residual is orthogonal to that span, so the Pythagorean theorem proves minimality and uniqueness. Its squared norm is $1/600$. A weight vanishing at a single endpoint still gives a positive-definite inner product on continuous functions: a nonzero continuous function remains nonzero on an interval where the weight is positive.

An orthogonal projection residual is the difference $r=x-P_{W}x$ between a vector and its orthogonal projection onto a linear subspace $W$. It is orthogonal to every vector in $W$, so $⟨r,w⟩=0$ for all $w\in W$. The Gram-Schmidt process uses such residuals to remove directions already represented by earlier basis vectors.

This is an optimality condition as well as a geometric description. No permitted change within $W$ can remove a further orthogonal component of $r$. In the preceding dot product example, $r=(2,-1{)}^T$ is perpendicular to the line spanned by $(1,2{)}^T$.

The selected inner product matters. Keep $x=(3,1{)}^T$ and the same line, but use the weighted Euclidean inner product with $M=\mathrm{diag}(4,1)$. The projection coefficient and residual becomeNow $4(r_M{)}_1{e}_1+(r_M{)}_2{e}_2=5-5=0$, although $r_M\cdot e=-15/4\ne 0$. The orthogonality is with respect to $M$, so it need not look perpendicular in the ordinary drawing. The squared weighted error isThe weighted optimum reduces the first-coordinate error from $2$ to $5/4$, at the cost of increasing the magnitude of the second-coordinate error from $1$ to $5/2$. The weights express which errors are more costly.

In linear least squares, residual orthogonality to the column space is expressed by the normal equations for linear least squares. With a weighted Euclidean inner product, the corresponding real-matrix equation is $A^{T}M(x-Ac)=0$. This makes the choice of error measurement part of the mathematical model, rather than merely a plotting convention.

A matrix represents a linear projection when applying it twice has the same effect as applying it once. For an orthogonal projection with the standard Hermitian inner product on ${\mathbb{C}}^m$, the additional condition is that the conjugate transpose equals the original matrix:Thus $P$ is both idempotent and a Hermitian matrix. Over the real numbers this says $P^2=P=P^T$. Its image of a linear map is the retained linear subspace, while its kernel of a linear map is the orthogonal complement: vectors in the image are unchanged, and vectors in the kernel become zero.

For a matrix $A$ with full column rank, projecting onto its column space givesThe invertibility of $A^{†}A$ follows from linear independence of the columns. If those columns are an orthonormal set, $A^{†}A=I$ and $P=A{A}^{†}$. These formulas and the conditions $P^2=P=P^T$ in the real case are developed in MIT 18.06SC, Projections onto Subspaces, pp. 1–3.

For the line spanned by $e=(1,2{)}^T$,In particular, $Pe=e$ and $P(2,-1{)}^T=0$. The eigenvalues are $1$ on the retained direction and $0$ on the removed direction. This exposes the meaning of an orthogonal projection as retaining selected components, rather than stretching them.

Symmetry refers to the specified inner product. In the real weighted setting, the formula becomes $P_M=A(A^{T}MA{)}^{-1}{A}^{T}M$ and satisfies $P_M^2=P_M$ and $P_M^{T}M=M{P}_M$. For the preceding weighted line example,It is not symmetric under the ordinary transpose, but it is an orthogonal projection for the weighted Euclidean inner product. The matrix criterion $P=P^{†}$ should therefore be used with the standard Hermitian inner product, not silently transferred to a different geometry.

Under orthogonal projection from one plane onto another, a circle of radius $R$ becomes an ellipse with semiaxes $R$ and $R|\cos \alpha |$, where $\alpha$ is the angle between the planes' unit normal vectors. To prove this, choose one in-plane unit vector parallel to the planes' intersection and the other perpendicular to it. The first vector is unchanged by the projection; the second has projected length $|\cos \alpha |$ and remains perpendicular to the first. Parametrizing the circle in that orthonormal basis therefore gives projected coordinates $(R\cos t,R\cos \alpha \sin t)$. Parallel planes have a circular image, and perpendicular planes give the degenerate case of a line segment. An arbitrary center simply projects to the center of the image.

For a nonnull vector $k$ in a space with a nondegenerate bilinear form, $P_{\mu \nu }={\eta }_{\mu \nu }-k_{\mu }{k}_{\nu }/k^2$ projects onto the subspace transverse to $k$. It obeys $P_{\mu \nu }{k}^{\nu }=0$ and $P^2=P$.

For orthogonal polynomials $Q_0,\dots ,Q_n$ under a positive weighted integral inner product, the least-squares approximation to $f$ of degree at most $n$ isIts residual is orthogonal to every polynomial of degree at most $n$.

Two vectors are orthogonal when their inner product is zero.

For real Euclidean vectors, the inner product is the dot product, so $(3,4)$ and $(4,-3)$ are orthogonal: $3(4)+4(-3)=0$. For nonzero Euclidean vectors, this means their angle is $90^{\circ }$. The zero vector is orthogonal to every vector, although it has no direction or defined angle.^[ref]

In an abstract inner product space, orthogonality always refers to its chosen inner product. For example, the weighted inner product $⟨u,v{⟩}_w=4u_1{v}_1+u_2{v}_2$ makes $(1,1)$ and $(1,-4)$ orthogonal, since $4-4=0$. Their ordinary dot product is $-3$, so their arrows are not perpendicular in the usual Cartesian coordinate system. The algebraic definition also applies to functions, where arrows need not be available.

A finite collection of nonzero pairwise orthogonal vectors is linearly independent. Given ${\sum }_i{a}_i{q}_i=0$, take the inner product with $q_j$ to obtain $a_j\Vert q_j{\Vert }^2=0$. Positive definiteness and $q_j\ne 0$ force $a_j=0$ for every index. The nonzero condition matters: adjoining a zero vector preserves pairwise orthogonality but destroys linear independence.

In the Hilbert space $L^2([0,2\pi ])$, choose orthonormal functions ${\varphi }_1(x)=\cos x/\sqrt{\pi }$ and ${\varphi }_2(x)=\sin x/\sqrt{\pi }$. Their inner product is zero, but for the real potential energy $V(x)=V_0\sin 2x$,Thus ordinary orthogonality alone does not remove an off-diagonal matrix element of a spatially varying potential energy. A constant scalar potential does give zero between distinct orthonormal states. The example separates these two claims; it is not a model of localized atomic orbitals in a particular crystal.

An orthogonal basis of an inner-product vector space consists of nonzero mutually orthogonal vectors that span the space. The orthogonal projection of $v$ onto a finite-dimensional span is ${\sum }_i⟨v,u_i⟩u_i/⟨u_i,u_i⟩$. Dividing each vector by its norm produces an orthonormal basis. For centered random variables, the inner product is covariance; an orthogonal basis therefore consists of uncorrelated variables, without requiring unit variance.

Two vectors are nonorthogonal when their inner product is nonzero. In the power method, the starting vector must be nonorthogonal to a dominant eigenvector.

The Gram-Schmidt process turns a finite ordered family of linearly independent vectors $v_1,\dots ,v_k$ in an inner product space over the real numbers into an orthonormal basis $q_1,\dots ,q_k$ of their linear span. It keeps every successive linear span, so the first $j$ new vectors describe exactly the same directions as the first $j$ inputs. Its idea is geometric: remove the orthogonal projection onto directions already found, then divide the remaining vector by its norm. This makes a complicated basis easier to use without changing the vector subspace it describes.^[ref]

Start with $u_1=v_1$ and $q_1=v_1/\Vert v_1\Vert$. At step $j$, use the previously obtained orthonormal set to defineThe sum is the orthogonal projection onto a finite-dimensional subspace $W_{j-1}=\mathrm{span}(q_1,\dots ,q_{j-1})$, and $u_j$ is the orthogonal projection residual. Linear independence ensures $u_j\ne 0$: otherwise $v_j$ would already be a linear combination of earlier inputs. If the input family is linearly dependent, use only the retained orthonormal vectors in the sum and omit an input whenever its remainder is the zero vector; do not divide by zero. The retained outputs still form an orthonormal basis of the original linear span, but their number can be smaller than $k$.

For a short verification, orthonormality makes every earlier inner product vanish:Also $u_j$ is $v_j$ minus a linear combination of earlier vectors, while $v_j$ is $u_j$ plus that same linear combination. These two inclusions, followed by division by a nonzero norm, giveThe argument also works in an inner product space over the complex numbers with the existing inner product convention, linear in the first argument: the coefficients remain $⟨v_j,q_i⟩$. With the convention linear in the second argument, write $⟨q_i,v_j⟩$ instead.

For a first picture, take $v_1=(1,1{)}^T$ and $v_2=(1,0{)}^T$ in ${\mathbb{R}}^2$ with the ordinary dot product. ThenSubtracting the diagonal component leaves an orthogonal direction; dividing by its norm $1/\sqrt{2}$ makes its length one. If the second input were $(2,2{)}^T$ instead, its orthogonal projection would be the entire input and $u_2=0$, revealing linear dependence.

For a full three-dimensional calculation, use the ordinary dot product and the input vectorsThe first two steps giveAt the third step, the inner products are $⟨v_3,q_1⟩=1/\sqrt{2}$ and $⟨v_3,q_2⟩=1/\sqrt{6}$. ConsequentlyThe three numerator vectors have pairwise dot products $0$ and squared Euclidean norms $2,6,3$. Thus the displayed outputs are indeed an orthonormal basis, and the three nonzero remainders also confirm linear independence of the inputs. Keeping the removed coefficients produces the QR decomposition below.