Orthogonal projection ($P$)

The orthogonal projection $P_F$ onto a closed subspace of a Hilbert space $F$ sends $x$ to its component in the orthogonal decompositionThe residual $r$ is orthogonal to every permitted direction. For any $z\in F$, the Pythagorean theorem givesThus $P_{F}x$ uniquely minimizes the distance to $F$. It is a linear map, is idempotent, and is nonexpansive: $\Vert P_{F}x-P_{F}y\Vert \le \Vert x-y\Vert$; taking $y=0$ gives $\Vert P_{F}x\Vert \le \Vert x\Vert$.

In ${\mathbb{R}}^2$, project $v=(1,3)$ onto the line spanned by $u=(2,1)$. The inner product givesThe check $⟨r,u⟩=-2+2=0$ confirms orthogonality, and $\Vert v{\Vert }^2=10=5+5=\Vert P_{F}v{\Vert }^2+\Vert r{\Vert }^2$. Projecting again changes nothing, since the result already lies on the line.

For a finite orthonormal basis $e_1,\dots ,e_m$ of $F$, the same operation is $P_{F}x={\sum }_j⟨x,e_j⟩e_j$, using the convention linear in the first argument. Replacing coordinate vectors by functions gives exactly the same approximation principle. For $f(x)=x^2$ in the real L2 space on $[-1,1]$ and $F=\mathrm{span}\{1,x\}$, the directions are orthogonal, with squared norms $2$ and $2/3$. Their inner products with $f$ are $2/3$ and zero, soDirect integration gives the squared error $\Vert x^2-1/3{\Vert }^2=8/45$. For any competing affine polynomial, the orthogonal residual yieldsEvery deviation from $a=1/3$, $b=0$ adds a nonnegative error. The best function need not pass through any selected data point: it minimizes an integrated error across the whole interval.

If idempotent matrices $P_1,\dots ,P_k$ satisfy $P_i{P}_j=0$ whenever $i\ne j$, then $({\sum }_i{P}_i{)}^2={\sum }_{i,j}{P}_i{P}_j={\sum }_i{P}_i$. Thus the sum is idempotent. For a real orthonormal set $q_i$, the matrices $P_i=q_i{q}_i^T$ meet this condition. Their sum is the orthogonal projector onto their span; the symmetric matrix property follows by transposing the sum. Arbitrary sums of idempotent matrices need not be idempotent.

The foot from $\mathbf{c}$ to the line $\mathbf{a}+t\mathbf{u}$, with $\mathbf{u}\ne 0$, is its orthogonal projection onto that line. Solving $(\mathbf{c}-\mathbf{a}-t\mathbf{u})\cdot \mathbf{u}=0$ gives the displayed formula and proves uniqueness. It belongs to the line segment with endpoints $\mathbf{a}$ and $\mathbf{a}+\mathbf{u}$ exactly when $0\le (\mathbf{c}-\mathbf{a})\cdot \mathbf{u}\le \Vert \mathbf{u}{\Vert }^2$.

For a nonzero closed subspace of a Hilbert space $V$ and a closed subspace of a Hilbert space $W$, this angle measures the worst loss under projection from $V$ to $W$. Its positive cosine is a uniform lower bound for $P_W{|}_V$. The order of the subspaces matters in general. This is distinct from the smallest angle between two subspaces, whose cosine is a supremum. For equal finite dimensions the cosine is the smallest singular value of the cross Gram matrix of orthonormal bases. When $V$ is zero, the infimum is over an empty unit sphere and does not define an angle in $[0,\pi /2]$; use a lower-bound formulation instead.

For closed subspaces of a Hilbert space, set $T=P_{V^{\perp }}{|}_W$. The two positive cosines bound $T$ and its adjoint operator $P_W{|}_{V^{\perp }}$ below. The first bound gives a closed range and injectivity; the second makes the range dense by image-kernel orthogonality for an adjoint. Thus $T$ is a bounded bijection, and $w=T^{-1}{P}_{V^{\perp }}f$ yields the unique direct sum decomposition $f=w+v$. In equal finite dimensions, a lower bound on $T$ alone suffices by the rank-nullity theorem.

For two nonzero finite-dimensional subspaces with projections $P,Q$, define $\cos \vartheta ={\sup }_{\Vert u\Vert =\Vert v\Vert =1}|⟨u,v⟩|=\Vert PQ\Vert$, with vectors drawn from the respective subspaces. The angle is zero when they intersect nontrivially. To measure separation after removing a common intersection, restrict both subspaces to its orthogonal complement. The resulting angle controls sums of positive operators in the Kitaev geometrical lemma. For a zero subspace, the vector supremum is empty; the projection-norm convention $\cos \vartheta =\Vert PQ\Vert =0$ is still usable when explicitly adopted.

Let $A,B$ be positive operators on a finite-dimensional Hilbert space, with every nonzero eigenvalue at least $\gamma >0$. Let $P,Q$ be the orthogonal projections onto $\ker A,\ker B$ and define $\cos \vartheta =\Vert PQ\Vert$. ThenWhen the kernels have no common nonzero vector, this gives a positive lower bound. The projection formula also covers a zero kernel without taking a supremum over an empty unit sphere.

Indeed, $A\ge \gamma (I-P)$ and $B\ge \gamma (I-Q)$. For $p\in \mathrm{ran}P$, $q\in \mathrm{ran}Q$, $|⟨p,q⟩|\le \Vert PQ\Vert \Vert p\Vert \Vert q\Vert$. The map $(p,q)\mapsto p+q$ consequently has squared operator norm at most $1+\Vert PQ\Vert$, so $P+Q\le (1+\Vert PQ\Vert )I$. Combining these inequalities proves the result. With common kernel $N$, restrict to $N^{\perp }$ and use the kernel projections there to bound the positive spectral gap.

In infinite dimension, replace the eigenvalue condition by the actual operator inequalities $A\ge \gamma (I-P)$ and $B\ge \gamma (I-Q)$. An assertion only about eigenvalues can be vacuous in the presence of continuous spectrum, and does not ensure this uniform gap.

For orthogonal projections $P,Q$ with $\Vert PQ\Vert \le q<1$, the identity $(PQ{)}^r=PQ(QPQ{)}^{r-1}$ and $\Vert QPQ\Vert =\Vert PQ{\Vert }^2$ imply $\Vert (PQ{)}^r\Vert \le q^{2r-1}$ for $r\ge 1$. When the projections have a common fixed subspace, first remove its orthogonal projection. This estimate is stronger than generic submultiplicativity and is useful for alternating local projection layers.