Inner product ($⟨\cdot ,\cdot ⟩$)

Averaging squared norms of all signed sums of finitely many vectors in an inner-product space cancels every mixed term. This equates the average to the sum of the individual squared norms, distinguishing Hilbert-space geometry from other sequence-space norms.

The Fischer inner product on polynomials is $⟨x^{\alpha },x^{\beta }{⟩}_F=\alpha !{\delta }_{\alpha \beta }$. Multiplication by $x_i$ is adjoint to differentiation by $x_i$. Therefore multiplication by $|x{|}^2$ is adjoint to the ordinary Laplacian, yielding the harmonic decomposition of homogeneous polynomials by finite-dimensional orthogonality.

For complex matrices, the inner product $⟨A,B{⟩}_F=\mathrm{tr}(A^{\ast }B)$ is conjugate-linear in its first argument. On real symmetric matrices it becomes $\mathrm{tr}(AB)$. In particular $⟨A,x{x}^T{⟩}_F=x^{T}Ax$, identifying quadratic forms with linear tests on matrices.

The discrete L2 inner product is the cell-volume-weighted Hermitian inner product on a uniform grid. Its associated norm is the discrete L2 norm. It provides the finite-dimensional analogue of spatial integration by parts and continuous energy method identities.

For a positive weight $w$, the weighted inner product of functions is $⟨f,g{⟩}_w=\int \overline{f}gw$. It is the usual inner product in the L2 space for the measure $w(x)dx$. A Sturm-Liouville eigenfunction expansion uses this weight to make its eigenfunctions orthogonal.

The dot product of Euclidean vectors is $\mathbf{a}\cdot \mathbf{b}={\sum }_i{a}_i{b}_i$. It equals $|\mathbf{a}||\mathbf{b}|\cos \theta$, where $\theta$ is the angle between them.

For $x,y\in {\mathbb{F}}_2^n$, the binary inner product is $x\cdot y={\sum }_i{x}_i{y}_i$ in ${\mathbb{F}}_2$. Its value is the parity of the coordinates at which both vectors are one.

An inner-product space is a vector space equipped with an inner product. The inner product induces the norm $\Vert x\Vert =\sqrt{⟨x,x⟩}$ and hence a metric.

An orthonormal set consists of vectors that have norm one and are pairwise orthogonal. Its Gram matrix is the identity matrix.

An orthonormal basis is a basis whose vectors are pairwise orthogonal and each have norm one.

For vectors $v_1,\dots ,v_n$, the Gram matrix isTwo finite vector families with the same Gram matrix are related by an isometry between their spans; in finite dimensions that isometry can be extended to a unitary map after completing orthonormal bases.

For a basis $(u_i)$ of a finite-dimensional inner-product space with conjugate-linear first argument, define $G_{ij}=(u_i,u_j)$ and ${\hat{u}}_k={\sum }_l(G^{-1}{)}_{lk}{u}_l$. Then $(u_i,{\hat{u}}_k)={\delta }_{ik}$ and $({\hat{u}}_j,{\hat{u}}_k)=(G^{-1}{)}_{jk}$. This is the inner-product representation of the dual basis; requiring the dual vectors to lie in the given span is essential.

For a matrix $G$ with independent standard normal entries, $G^{T}G/n$ is a Gaussian empirical Gram matrix and has expectation equal to the identity. With mean known to be zero it is the raw empirical second moment. Subtracting a sample mean would give a different estimator. On any fixed unit vector its quadratic form is a chi-squared variable divided by $n$.

For $G$ with $n$ independent standard Gaussian rows in dimension $k$, combine a constant-radius sphere metric net, the quadratic form net bound, the chi-squared concentration inequality, and a union bound. This gives $P(\Vert G^{T}G/n-I_k{\Vert }_{\mathrm{op}}>1/2)\le 2\exp (k\log B-n/1024)$ for a numerical $B$. Taking $n$ larger than a constant times $k\log p$ gives exponential decay in $k\log p$. The subspace is fixed, so no union over supports is needed.

If $E=A^{T}A/d-I$ has entrywise maximum norm at most $t$, every vector $z$ supported on at most $s$ coordinates satisfies $|z^{T}Ez|\le t\Vert z{\Vert }_1^2\le st\Vert z{\Vert }_2^2$. Thus the squared norm distortion is at most $st$ on all such sparse vectors. Use the quadratic inequality at $z=0$, where a ratio would be undefined.

If $u$ and $v$ are orthogonal, then

If $e_0,\dots ,e_n$ is an orthogonal basis of a finite-dimensional subspace $W$, the unique closest point in $W$ to $f$ isThe residual $f-P_{W}f$ is orthogonal to $W$.

A complex matrix $P$ represents an orthogonal projection exactly whenIts image and kernel are orthogonal complements.

For a nonnull vector $k$ in a space with a nondegenerate bilinear form, $P_{\mu \nu }={\eta }_{\mu \nu }-k_{\mu }{k}_{\nu }/k^2$ projects onto the subspace transverse to $k$. It obeys $P_{\mu \nu }{k}^{\nu }=0$ and $P^2=P$.

For orthogonal polynomials $Q_0,\dots ,Q_n$ under a positive weighted integral inner product, the least-squares approximation to $f$ of degree at most $n$ isIts residual is orthogonal to every polynomial of degree at most $n$.

Two vectors are orthogonal when their inner product is zero.

An orthogonal basis of an inner-product vector space consists of nonzero mutually orthogonal vectors that span the space. The orthogonal projection of $v$ onto a finite-dimensional span is ${\sum }_i⟨v,u_i⟩u_i/⟨u_i,u_i⟩$. Dividing each vector by its norm produces an orthonormal basis. For centered random variables, the inner product is covariance; an orthogonal basis therefore consists of uncorrelated variables, without requiring unit variance.

Two vectors are nonorthogonal when their inner product is nonzero. In the power method, the starting vector must be nonorthogonal to a dominant eigenvector.

The Gram-Schmidt process replaces linearly independent vectors $v_1,\dots ,v_n$ by orthogonal vectors with the same successive spans, subtracting from each $v_k$ its projections onto the preceding vectors.

A reduced QR decomposition of a full-column-rank matrix writes $A=QR$, where $Q$ has orthonormal columns and $R$ is upper triangular with positive diagonal.

For a full-column-rank $n\times k$ matrix $V$, extend the thin QR decomposition $V=Q_1{R}_1$ to an orthogonal matrix $Q=(Q_1{Q}_2)$. Thenso $Q^T$ maps the column space of $V$ onto the first $k$ coordinate directions. Successive Householder transformations construct the same reduction without first forming a full orthogonal basis.

A linear least-squares problem minimizes $\Vert Ax-b{\Vert }_2$ over $x$. If $A=QR$ has full column rank, orthogonality reduces it to the upper-triangular system formed from the leading block of $R$ and the leading entries of $Q^{T}b$.

For a symmetric bilinear form $B$, the same projection formulaproduces a $B$-orthogonal basis whenever the chosen pivots $B(u_j,u_j)$ are nonzero. Reordering the input vectors can avoid a zero pivot when a suitable anisotropic vector remains.