Orthogonal projection ($P$)

The orthogonal projection $P_F$ onto a closed subspace of a Hilbert space $F$ sends $x$ to its component in the orthogonal decompositionThe residual $r$ is orthogonal to every permitted direction. For any $z\in F$, the Pythagorean theorem givesThus $P_{F}x$ uniquely minimizes the distance to $F$. It is a linear map, is idempotent, and is nonexpansive: $\Vert P_{F}x-P_{F}y\Vert \le \Vert x-y\Vert$; taking $y=0$ gives $\Vert P_{F}x\Vert \le \Vert x\Vert$.

In ${\mathbb{R}}^2$, project $v=(1,3)$ onto the line spanned by $u=(2,1)$. The inner product givesThe check $⟨r,u⟩=-2+2=0$ confirms orthogonality, and $\Vert v{\Vert }^2=10=5+5=\Vert P_{F}v{\Vert }^2+\Vert r{\Vert }^2$. Projecting again changes nothing, since the result already lies on the line.

For a finite orthonormal basis $e_1,\dots ,e_m$ of $F$, the same operation is $P_{F}x={\sum }_j⟨x,e_j⟩e_j$, using the convention linear in the first argument. Replacing coordinate vectors by functions gives exactly the same approximation principle. For $f(x)=x^2$ in the real L2 space on $[-1,1]$ and $F=\mathrm{span}\{1,x\}$, the directions are orthogonal, with squared norms $2$ and $2/3$. Their inner products with $f$ are $2/3$ and zero, soDirect integration gives the squared error $\Vert x^2-1/3{\Vert }^2=8/45$. For any competing affine polynomial, the orthogonal residual yieldsEvery deviation from $a=1/3$, $b=0$ adds a nonnegative error. The best function need not pass through any selected data point: it minimizes an integrated error across the whole interval.

For a nonzero closed subspace of a Hilbert space $V$ and a closed subspace of a Hilbert space $W$, this angle measures the worst loss under projection from $V$ to $W$. Its positive cosine is a uniform lower bound for $P_W{|}_V$. The order of the subspaces matters in general. This is distinct from the smallest angle between two subspaces, whose cosine is a supremum. For equal finite dimensions the cosine is the smallest singular value of the cross Gram matrix of orthonormal bases. When $V$ is zero, the infimum is over an empty unit sphere and does not define an angle in $[0,\pi /2]$; use a lower-bound formulation instead.

For closed subspaces of a Hilbert space, set $T=P_{V^{\perp }}{|}_W$. The two positive cosines bound $T$ and its adjoint operator $P_W{|}_{V^{\perp }}$ below. The first bound gives a closed range and injectivity; the second makes the range dense by image-kernel orthogonality for an adjoint. Thus $T$ is a bounded bijection, and $w=T^{-1}{P}_{V^{\perp }}f$ yields the unique direct sum decomposition $f=w+v$. In equal finite dimensions, a lower bound on $T$ alone suffices by the rank-nullity theorem.

For two finite-dimensional subspaces with projections $P,Q$, define $\cos \vartheta ={\sup }_{\Vert u\Vert =\Vert v\Vert =1}|⟨u,v⟩|=\Vert PQ\Vert$, with vectors drawn from the respective subspaces. The angle is zero when they intersect nontrivially. To measure separation after removing a common intersection, restrict both subspaces to its orthogonal complement. The resulting angle controls sums of positive operators in the Kitaev geometrical lemma.

Suppose positive operators $A,B$ have positive eigenvalues at least $\gamma$ and their nullspaces meet only at zero. If $\vartheta$ is their smallest angle between two subspaces, then $A+B\ge 2\gamma {\sin }^2(\vartheta /2)I$. Indeed $A+B\ge \gamma (2I-P-Q)$, while $(P+Q{)}^2\le (1+\cos \vartheta )(P+Q)$ gives $P+Q\le (1+\cos \vartheta )I$. With a common nullspace, apply the same proof on its orthogonal complement to bound the positive spectral gap.

For orthogonal projections $P,Q$ with $\Vert PQ\Vert \le q<1$, the identity $(PQ{)}^r=PQ(QPQ{)}^{r-1}$ and $\Vert QPQ\Vert =\Vert PQ{\Vert }^2$ imply $\Vert (PQ{)}^r\Vert \le q^{2r-1}$ for $r\ge 1$. When the projections have a common fixed subspace, first remove its orthogonal projection. This estimate is stronger than generic submultiplicativity and is useful for alternating local projection layers.