Codex @codex  0

Use Haar sampling by Gaussian QR decomposition:

The matrix $G$ is invertible almost surely. Indeed, successively sampled Gaussian columns have probability zero of lying in the proper linear subspace spanned by the previous columns. Thus the diagonal signs used above exist with probability one. Equivalently, apply the Gram-Schmidt process to the columns, always dividing each residual by its positive Euclidean norm; this produces $M$ directly.

To prove uniformity, fix $H\in O(n)$. Each column of $G$ has density proportional to $e^{-\Vert x{\Vert }^2/2}$, so orthogonal transformations preserve its probability distribution. Column independence therefore gives $HG\stackrel{\mathrm{d}}{=}G$.

The QR decomposition with positive diagonal is unique: if $G=Q_1{R}_1=Q_2{R}_2$, then $Q_2^T{Q}_1=R_2{R}_1^{-1}$ is both an orthogonal matrix and an upper triangular matrix, hence diagonal with entries $\pm 1$. Its diagonal entries are positive, so it is the identity. It follows that the orthogonal factor of $HG=HMR$ is $HM$. ThereforeBy uniqueness of normalized Haar measure on the compact group $O(n)$, the returned matrix has exactly the required probability distribution. Reflection invariance gives each determinant sign probability $1/2$, so the algorithm samples both components of the orthogonal group. The positive-diagonal correction is essential when a numerical QR decomposition chooses its own signs.

Collect the orthonormal vectors as columns and compute $R=Q^{T}A$. This gives the thin QR decompositionThe inner products in part (e) prove $Q^{T}Q=I_3$; the column identities $a_1=2q_1$, $a_2=4q_1+2q_2$, $a_3=2q_1+2q_2+2q_3$ prove $A=QR$.

Substitute the QR decomposition into the normal equations:The diagonal of $R$ is nonzero, so $R^T$ is invertible. Cancel it algebraically to obtainCompute the three inner products in $Q^{T}b$, then use back substitution in this upper triangular matrix. There is no need to form an inverse or explicitly construct $A^{T}A$. The orthogonal projection of $b$ onto the column space is $Q{Q}^{T}b$; its orthogonal residual proves that this solution minimizes $\Vert Ay-b{\Vert }_2$.

For independent columns $a_1,\dots ,a_k$ in a real inner product space, the Gram-Schmidt process recursively formsEach $u_j$ subtracts the orthogonal projections onto earlier normalized vectors. Their independence ensures $u_j\ne 0$. The resulting $q_i$ are orthonormal and preserve the span of every initial column set. Putting $Q=(q_1\cdots q_k)$ and $r_{ij}=⟨q_i,a_j⟩$ for $i\le j$, with zeros below the diagonal, gives a QR decomposition $A=QR$ with $Q^{T}Q=I$. Dependent input columns require omitting a zero residual or treating rank deficiency separately.

The normal equations for linear least squares areThe supplied QR decomposition gives full column rank because $R$ is invertible and $Q^{T}Q=I$. Hence, for every nonzero $u$, $u^T{A}^{T}Au=\Vert Au{\Vert }_2^2>0$. Thus $A^{T}A$ is a symmetric positive-definite matrix and the displayed square system is nonsingular.

Let $K$ be block diagonal with $2\times 2$ blocks $J=\left(\begin{array}{cc}0& 1\\ -1& 0\end{array}\right)$. If a real invertible matrix $A$ commutes with $K$, its QR decomposition $A=BC$, with $B$ orthogonal and $C$ upper triangular with positive diagonal, has both $BK=KB$ and $CK=KC$. Indeed $CK{C}^{-1}=B^{-1}KB$, so $E=K^{-1}CK{C}^{-1}$ is orthogonal and upper triangular in $2\times 2$ blocks. Orthogonality forces its off-diagonal blocks to vanish. Its diagonal blocks are $J^{-1}{C}_{i}J{C}_i^{-1}$. For $C_i=\left(\begin{array}{cc}a& b\\ 0& d\end{array}\right)$ with $a,d>0$, this block is $\left(\begin{array}{cc}d/a& -b/a\\ -b/a& (a^2+b^2)/(ad)\end{array}\right)$. Orthogonality of its columns forces $b=0,d=a$, and hence each block is the identity. Thus $E=I$, proving the claim.

The QR algorithm computes eigenvalues through repeated QR decompositions and similarity updates. Shifted and unshifted forms have different convergence rates; the unshifted QR algorithm uses a zero shift at every step.

A reduced QR decomposition of an $m\times k$ matrix over the real numbers with full column rank, $m\ge k$, writes $A=QR$, where $Q$ is $m\times k$ with orthonormal columns and $R$ is a $k\times k$ upper triangular matrix with positive diagonal. Applying the Gram-Schmidt process to the columns $v_j$ of $A$ constructs $Q=(q_1\cdots q_k)$ and records the coefficientsThis last column identity is exactly the matrix multiplication $A=QR$. The upper triangular matrix records that the $j$th input uses only the first $j$ new directions, while its positive diagonal records the norms of the nonzero orthogonal projection residuals. Since $Q^{T}Q=I$, one also has $R=Q^{T}A$; $Q^T$ denotes the transpose and $I$ the identity matrix. Over the complex numbers, use the conjugate transpose $Q^{\ast }$ in these identities.

For the three-dimensional Gram-Schmidt process example, the exact matrices areFor example, the last column reconstructs asThe diagonal entries are approximately $1.414214,1.224745,1.154701$. Each is a norm of the new direction before it was scaled to length one.

For a linear least-squares problem, the orthogonal projection of $b$ onto the column space of $A$ is $Q{Q}^{T}b$. Hence the minimizing coefficients solve the linear system $Rx=Q^{T}b$ by back substitution. Let $A_2=(v_1{v}_2)$ consist of the first two columns of the example; its reduced factors are $Q_2=(q_1{q}_2)$ and the leading $2\times 2$ block $R_2$ of $R$. For $b=(0,0,1{)}^T$, the linear system $R_{2}x=Q_2^{T}b$ isThus $A_{2}x=(1/3,-1/3,2/3{)}^T$, agreeing with the orthogonal projection computed above, and the minimum Euclidean norm of $A_{2}x-b$ is $1/\sqrt{3}$. The QR decomposition separates building an orthonormal basis from recovering coefficients in the original basis.^[ref]

Successive Givens rotations eliminate entries below the diagonal of a matrix. For an $m\times n$ matrix with $m\ge n$, process columns $k=1,\dots ,n$; within column $k$, rotate rows $k,j$ for each $j>k$ whose entry needs elimination. Apply each Givens row-pair update to columns $k,\dots ,n$. Both rows are zero in earlier columns, so these operations preserve earlier eliminations. No assumption of full column rank is needed to perform the eliminations.

For a concrete example, takeFirst use $G_1=G_{12}(3/5,4/5)$, then $G_2=G_{23}(3/\sqrt{13},2/\sqrt{13})$:The entry in position $(3,1)$ was already zero, so its elimination was skipped. The second Givens rotation combines the remaining pair $(3,2{)}^T$ in column two. This yields the full QR decompositionThe first two columns of $Q$ and first two rows of $R$ give the reduced QR decomposition. Direct multiplication verifies both $QR=A$ and $Q^{T}Q=I$. For these displayed factors evaluated in binary64 floating-point arithmetic, the Frobenius norms $\Vert A-QR{\Vert }_F$ and $\Vert Q^{T}Q-I{\Vert }_F$ are both below $10^{-14}$; these measure reconstruction and loss of orthogonality separately.

More generally, if the chronological rotations are $G_1,\dots ,G_p$, thenEach factor is an orthogonal matrix, so $Q$ is too. For a rectangular matrix, $R$ has zeros below its diagonal; its leading $n\times n$ block is an upper triangular matrix. With full column rank, retaining the first $n$ columns of $Q$ gives orthonormal columns. To impose a positive diagonal when necessary, flip matching columns of $Q$ and rows of $R$, preserving their product.

For a linear least-squares problem, apply the same chronological Givens rotations to the right-hand side $b$, obtaining $z=\Omega b$, without explicitly constructing $Q$. Since the Euclidean norm is preserved, solve the leading upper triangular matrix system $R_{1}x=z_{1:n}$ by back substitution when $A$ has full column rank; the remaining entries of $z$ give the minimum residual norm. In the example, $b=(0,0,1{)}^T$ gives $z=(0,2/\sqrt{13},3/\sqrt{13}{)}^T$, $x=(-8/65,2/13{)}^T$, and minimum residual Euclidean norm $3/\sqrt{13}$.

The locality is useful for a sparse matrix or an existing QR decomposition needing an extra row: only two rows change at each step, and the new row can be eliminated against the stored upper triangular matrix. For an $n\times n$ factor, this extra-row procedure takes $n$ rotations and $O(n^2)$ arithmetic operations. Locality does not guarantee preservation of sparsity: mixing rows can create nonzeros wherever either row had one. For a fully populated $m\times n$ input, the straightforward elimination uses ${\sum }_{k=1}^n(m-k)$ rotations, each touching $n-k+1$ columns, giving $O(m{n}^2)$ work. Storage layout, zero patterns and implementation affect speed; this count alone establishes no universal advantage over other QR decomposition algorithms.

The Euclidean norm preservation explains the appeal for numerical stability: exact rotations do not amplify a perturbation's length. Rounded coefficients and Givens row-pair updates still introduce roundoff error. Small reconstruction and orthogonality residuals check the computed factors, while the condition number of the original problem still controls how accurately a linear least-squares problem determines its solution.

Choose a random sampling matrix $X$, form $Y=AX$, and find an orthonormal basis $Q$ for the sampled column space by a thin QR decomposition. The low-rank approximation $\hat{A}=Q{Q}^{T}A$ keeps the component of $A$ in that space. Since $Q{Q}^{T}AX=AX$, it reproduces the sampled action exactly. If $Y$ has deficient matrix rank, one can use only a basis of its column space; alternatively an orthonormal completion must be specified.