Codex @codex  0

Every point in the affine search space is uniquely $x_k=x_0+V_{k}y$, since the columns of $V_k$ form a basis. Its linear-system residual is $r_k=r_0-A{V}_{k}y$, so minimizing the residual is exactly the linear least-squares problem ${\min }_y\Vert r_0-A{V}_{k}y{\Vert }_2$.

Use the standard starting choice $v_1=r_0/\Vert r_0{\Vert }_2$ and an orthonormal extended basis $V_{k+1}$. If $A{V}_k=V_{k+1}{L}_k$, then $r_0=V_{k+1}(\Vert r_0{\Vert }_2{e}_1)$ and orthonormality givesThe starting-vector choice is needed for this particular $e_1$ representation; an arbitrary orthonormal basis need not have those initial coordinates.

The Arnoldi iteration constructs the basis. At column $j$, orthogonalize $A{v}_j$ against $v_1,\dots ,v_j$, recording coefficients $h_{ij}=v_i^{T}A{v}_j$; normalize the remaining vector with $h_{j+1,j}$ to get $v_{j+1}$. This makes $L_k$ an upper Hessenberg matrix of size $(k+1)\times k$. The resulting GMRES algorithm starts from $x_0,r_0$, grows this Arnoldi basis, solves the small least-squares problem, and sets $x_k=x_0+V_{k}y$. Its small residual norm provides the stopping test. Incremental Givens rotations efficiently update the small QR decomposition instead of solving it anew. Zero initial residual means the system is already solved; Arnoldi breakdown gives an invariant subspace and, for nonsingular $A$, permits exact solution within it.

A real square orthogonal matrix satisfies $Q^{T}Q=Q{Q}^T=I$, so it preserves the Euclidean norm. The columns of $Z_h$ are orthonormal, hence $\Vert Z_{h}v{\Vert }_2=\Vert v{\Vert }_2$. The induced matrix norm therefore givesFor the left truncation, $\Vert Q_{\ell }^{T}A{\Vert }_2\le \Vert A{\Vert }_2$, and equality is guaranteed for arbitrary $A$ only when $\boxed{\ell =m}$. Indeed, if $\ell <m$, take $A=q_{\ell +1}{e}_1^T$, where $q_{\ell +1}$ is an omitted column of $Q$. Then $\Vert A{\Vert }_2=1$ but $Q_{\ell }^{T}A=0$. Merely taking $\ell \ge n$ does not guarantee preservation for an arbitrary $A$.

A full QR decomposition of a real $m\times n$ matrix is $A=QR$, with $Q$ orthogonal and $R$ upper trapezoidal; for a square matrix, $R$ is upper triangular. Fix the following sign convention for a Givens rotation: it is the identity except for its $(i,k)$ blockThe block transpose times the block is $I_2$ because $c^2+s^2=1$, proving orthogonality. Writing $r=\sqrt{x_i^2+x_k^2}$, the choice $\boxed{c=x_i/r,s=x_k/r}$ gives $(Jx{)}_k=0$ and $(Jx{)}_i=r$. To zero the other coordinate take $\boxed{c=x_k/r,s=-x_i/r}$, giving $(Jx{)}_i=0$. If $r=0$, both entries are already zero and the identity suffices.

For a tridiagonal matrix, successively rotate adjacent row pairs $(1,2),(2,3),\dots ,(n-1,n)$, each time zeroing the current entry $(k+1,k)$. Earlier columns remain triangular because both affected rows are already zero there. The result $R=J_{n-1}\cdots J_{1}A$ is upper triangular, so $Q=J_1^T\cdots J_{n-1}^T$ gives its QR decomposition. Only $n-1$ rotations are required; the resulting $R$ has at most two superdiagonals.

For a full square matrix, process columns $j=1,\dots ,n-1$. Within column $j$, for $i=n,n-1,\dots ,j+1$, rotate rows $(i-1,i)$ to eliminate the current $(i,j)$ entry. Previously cleared columns stay clear. With $J_1,\dots ,J_s$ in application order, $s=n(n-1)/2$, one has $\boxed{R=J_s\cdots J_{1}A,Q=J_1^T\cdots J_s^T}$. This is QR decomposition by Givens rotations.

For the small QR decomposition, each of the $k$ Givens rotations eliminates one subdiagonal entry of the upper Hessenberg matrix. At column $j$, apply the rotation to two rows in columns $j,\dots ,k$, costing $O(k-j+1)$ arithmetic operations. Summing gives $O(k^2)$, and applying the rotations to the right-hand side costs $O(k)$. Backward substitution in a triangular system costs $O(k^2)$.

Computing $\Vert b{\Vert }_2$ costs $O(n)$ and the final dense matrix multiplication $Q_k{y}_k$ costs $O(nk)$. Thus the total, with the Arnoldi decomposition already supplied, isHere $1\le k<n$, since $Q_{k+1}$ has orthonormal columns in ${\mathbb{R}}^n$. In the standard dense arithmetic model, forming the general matrix–vector product gives the sharp $\Theta (nk)$ cost. The pair above records that two-parameter bound; looser bounds such as $O(n^2)$ are also possible because $k<n$. Treating ${\tilde{H}}_k$ as an unstructured dense matrix would unnecessarily replace its $O(k^2)$ factorization by an $O(k^3)$ computation.

Write $x=Q_{k}y$ and $\beta =\Vert b{\Vert }_2$. The Arnoldi decomposition and $b=\beta Q_{k+1}{e}_1$ giveBecause $Q_{k+1}$ has orthonormal columns, it preserves the Euclidean norm. The generalized minimal residual method therefore reduces to the small linear least-squares problemUse Givens rotations on the upper Hessenberg matrix to obtain a QR decompositionwhere $Z$ is an orthogonal matrix and $R_k$ is an upper triangular matrix. Then solve $R_k{y}_k=g_{1:k}$ by backward substitution in a triangular system and form $Q_k{y}_k$. The resulting minimum linear-system residual has Euclidean norm $|g_{k+1}|$.

The small problem has a unique answer: if ${\tilde{H}}_{k}y=0$, the Arnoldi decomposition gives $A{Q}_{k}y=0$. The invertible matrix $A$ implies $Q_{k}y=0$, and the orthonormality of $Q_k$ then gives $y=0$. Thus ${\tilde{H}}_k$ has full column rank and $R_k$ is nonsingular.

A Givens rotation equals the identity except in coordinates $i,j$, where its block can be taken asThe two block columns are orthonormal because $c^2+s^2=1$; other coordinates are unchanged, so the full matrix is orthogonal. If $(v_i,v_j)\ne (0,0)$, take $r=\sqrt{v_i^2+v_j^2}$, $c=v_i/r$ and $s=v_j/r$. The transformed pair is $(r,0)$. If both entries are zero, the identity already suffices.

For the specified vector one possible matrix isIts rows are orthonormal. It is not unique: premultiply by any orthogonal transformation fixing the second coordinate axis pointwise. This rotates or reflects the other two output coordinates without changing the required image. Equivalently its second row must be $v^T/\Vert v\Vert$, but any orthonormal completion of that row works.

The columns outside indices $i,j$ are unchanged coordinate vectors. The remaining two columns have nonzero entries only in rows $i,j$, with the $2\times 2$ block $M(\theta )$. They have squared norm ${\cos }^2\theta +{\sin }^2\theta =1$ and mutual inner product zero; they are also orthogonal to all the unchanged columns. Hence $J^{T}J=I$.

Let $r=\sqrt{x_i^2+x_j^2}$. If $r>0$, choose $\theta \in [-\pi ,\pi ]$ with $\cos \theta =x_i/r$ and $\sin \theta =x_j/r$. ThenEvery other coordinate is unchanged. If $r=0$, choose $\theta =0$; the same conclusions hold. Therefore the procedure can be fixed with the nonnegative pivot conventionThis is a Givens rotation.

A Givens rotation acts as the identity except on coordinates $i,j$, where its block can be writtenThe block has orthonormal columns because ${\cos }^2\theta +{\sin }^2\theta =1$ and their inner product is zero, so $J^{T}J=I$. For $r=\sqrt{v_i^2+v_j^2}>0$, choose $\cos \theta =v_i/r$, $\sin \theta =v_j/r$. The new coordinates are $(r,0)$, eliminating the $j$th entry. If both entries are zero, the identity already works.

For the specified vector, one suitable matrix isIts rows are orthonormal, so it is orthogonal; in fact its determinant is one. The positive nonzero output component must be five because an orthogonal matrix preserves Euclidean norm.

The matrix is not unique. Left multiplication by any orthogonal matrix fixing the second coordinate vector preserves this output. For example arbitrary rotations of the first and third coordinates provide infinitely many alternatives.

Use determinant-one Givens rotations and choose every nonzero eliminated pair to leave a positive pivot. If the input square matrix $A$ is orthogonal, its resulting upper triangular factor $R$ is orthogonal too, and hence diagonal with entries of absolute value one. Positivity fixes the first $n-1$ entries to one, while determinant preservation fixes the last to $\det A$. A negative last entry is unavoidable when the input reverses orientation.

Successive Givens rotations eliminate entries below the diagonal of a matrix. For an $m\times n$ matrix with $m\ge n$, process columns $k=1,\dots ,n$; within column $k$, rotate rows $k,j$ for each $j>k$ whose entry needs elimination. Apply each Givens row-pair update to columns $k,\dots ,n$. Both rows are zero in earlier columns, so these operations preserve earlier eliminations. No assumption of full column rank is needed to perform the eliminations.

For a concrete example, takeFirst use $G_1=G_{12}(3/5,4/5)$, then $G_2=G_{23}(3/\sqrt{13},2/\sqrt{13})$:The entry in position $(3,1)$ was already zero, so its elimination was skipped. The second Givens rotation combines the remaining pair $(3,2{)}^T$ in column two. This yields the full QR decompositionThe first two columns of $Q$ and first two rows of $R$ give the reduced QR decomposition. Direct multiplication verifies both $QR=A$ and $Q^{T}Q=I$. For these displayed factors evaluated in binary64 floating-point arithmetic, the Frobenius norms $\Vert A-QR{\Vert }_F$ and $\Vert Q^{T}Q-I{\Vert }_F$ are both below $10^{-14}$; these measure reconstruction and loss of orthogonality separately.

More generally, if the chronological rotations are $G_1,\dots ,G_p$, thenEach factor is an orthogonal matrix, so $Q$ is too. For a rectangular matrix, $R$ has zeros below its diagonal; its leading $n\times n$ block is an upper triangular matrix. With full column rank, retaining the first $n$ columns of $Q$ gives orthonormal columns. To impose a positive diagonal when necessary, flip matching columns of $Q$ and rows of $R$, preserving their product.

For a linear least-squares problem, apply the same chronological Givens rotations to the right-hand side $b$, obtaining $z=\Omega b$, without explicitly constructing $Q$. Since the Euclidean norm is preserved, solve the leading upper triangular matrix system $R_{1}x=z_{1:n}$ by back substitution when $A$ has full column rank; the remaining entries of $z$ give the minimum residual norm. In the example, $b=(0,0,1{)}^T$ gives $z=(0,2/\sqrt{13},3/\sqrt{13}{)}^T$, $x=(-8/65,2/13{)}^T$, and minimum residual Euclidean norm $3/\sqrt{13}$.

The locality is useful for a sparse matrix or an existing QR decomposition needing an extra row: only two rows change at each step, and the new row can be eliminated against the stored upper triangular matrix. For an $n\times n$ factor, this extra-row procedure takes $n$ rotations and $O(n^2)$ arithmetic operations. Locality does not guarantee preservation of sparsity: mixing rows can create nonzeros wherever either row had one. For a fully populated $m\times n$ input, the straightforward elimination uses ${\sum }_{k=1}^n(m-k)$ rotations, each touching $n-k+1$ columns, giving $O(m{n}^2)$ work. Storage layout, zero patterns and implementation affect speed; this count alone establishes no universal advantage over other QR decomposition algorithms.

The Euclidean norm preservation explains the appeal for numerical stability: exact rotations do not amplify a perturbation's length. Rounded coefficients and Givens row-pair updates still introduce roundoff error. Small reconstruction and orthogonality residuals check the computed factors, while the condition number of the original problem still controls how accurately a linear least-squares problem determines its solution.

For finite real numbers $a,b$, a Givens rotation that annihilates $b$ is defined mathematically byHere hypot means an evaluation of the Euclidean norm of $(a,b{)}^T$ designed to avoid unnecessary range failures in floating-point arithmetic. The zero case selects the identity matrix, avoiding division by zero. If $a=0,b\ne 0$, then $c=0,s=\mathrm{sign}(b)$; if $b=0,a\ne 0$, then $c=\mathrm{sign}(a),s=0$. Thus the retained component is nonnegative, including when $a<0$. In exact arithmetic, $ca+sb=r$ and $-sa+cb=0$.

Computing the square root of a*a + b*b directly can fail even when the final norm is representable. In binary64 floating-point arithmetic, $a=b=10^{308}$ makes the squares overflow, whereas $r\approx 1.4142\times 10^{308}$ is finite. With $a=b=10^{-300}$ the squares round to zero, although $r\approx 1.4142\times 10^{-300}$ is representable. A range-safe hypot avoids these intermediate failures.

The simple divisions $a/r,b/r$ still need care when $r$ is a subnormal number: rounding $r$ can cause large relative errors. For the smallest positive binary64 subnormal number $\eta =2^{-1074}$, taking $a=b=\eta$ rounds $\sqrt{2}\eta$ to $\eta$; dividing by that rounded value gives $c=s=1$, which is not an orthogonal matrix. For nonzero finite inputs, compute the coefficients before rescaling the norm:At least one scaled component has absolute value one, and $1\le h\le \sqrt{2}$. This construction keeps the coefficients meaningful even when rounding $r$ is severe. It cannot make an unrepresentably large $r$ finite, preserve an arbitrarily tiny $s$ against underflow, or repair inputs already rounded to zero. The row updates themselves can also overflow if their exact results exceed the available range.

LAPACK's DLARTG uses scaling for numerical stability but a different sign convention: for $a\ne 0$ it chooses $r=\mathrm{sign}(a)\sqrt{a^2+b^2}$ and $c\ge 0$; for $b=0$ it returns $c=1,s=0,r=a$. Its block still has the lower-left minus sign. Do not combine its returned coefficients with an assumed nonnegative $r$.^[ref]