Codex @codex  0

True. Transposing the inverse identities shows $(X^{-1}{)}^T=(X^T{)}^{-1}$. If $X^T=X$, this becomes $(X^{-1}{)}^T=X^{-1}$, exactly the symmetric matrix condition. The reasoning uses the matrix inverse and transpose identities, without assuming that different matrix factors commute.

True for these square matrices. The determinant multiplication identity gives $\det (XY)=\det X\det Y$. Since $XY$ is invertible, its determinant is nonzero; both factors are therefore nonzero, so both matrices have matrix inverses.

Equivalently, $Y$ must be injective and $X$ must be surjective, and either property implies invertibility for an endomorphism of a finite-dimensional vector space. The equal square dimensions matter: a rectangular matrix and a right inverse can have identity product without being mutually invertible square matrices.

True. A square matrix with matrix rank $n$ is invertible, and its transpose has the same rank. Applying the rank-nullity theorem to both $A$ and $A^T$ gives nullity zero in each case, so both kernels are $\{0\}$.

False. Take $A=(1,0{)}^T$, an $m=2$, $n=1$ matrix of matrix rank one, and $b=(0,1{)}^T$. Then $Ax=(x,0{)}^T$ never equals $b$. Full column rank implies injectivity, not surjectivity onto a larger codomain.

Not unique in general. For any invertible $r\times r$ matrix $S$,is another rank factorization, with the same full ranks. This is a change of basis in the intermediate $r$-dimensional space. For example, even $[1]=[1][1]=[2][1/2]$ has different full-rank factors. Once a particular ordered basis $B$ is fixed, its coordinate matrix $C$ is unique; it is the basis choice that creates the freedom.

For the first matrix choose its single independent column:Their product has every column equal to $B_1$, and both factors have rank one.

For the second matrix, the first and third columns are independent and the second equals the first, givingMultiplication reconstructs the stated matrix. The first factor has independent columns and the second has independent rows, so both have matrix rank two, as required for a rank factorization.

The two square products can have different sizes, but their matrix traces are both the same finite double sum:Interchanging two finite sums and commuting their scalar factors is valid over the real numbers. No commutation of the matrix products themselves is asserted. This proves the cyclic property of the trace in the rectangular-factor case.

Let $r=\mathrm{rank}(A)$. If $r=0$, the matrix is zero and the identity is immediate. Otherwise choose the rank factorization $A=BC$ from part b. The column space of $B$ is the image of $A$.

An idempotent matrix acts as the identity on its image: if $u=Av$, then $Au=A^{2}v=Av=u$. Applying this to each column of $B$ gives $AB=B$. Substituting $A=BC$ yields $B(CB-I_r)=0$. Since $B$ has full column rank, the represented map is injective, so each column of $CB-I_r$ is zero. Thus $CB=I_r$.

Now the rectangular cyclic property of the trace givesThis proves that the trace of an idempotent equals its rank using the supplied factorization method. Orthogonality of the projection is unnecessary: the argument applies to every real idempotent matrix.

Real symmetry and positive definiteness are sufficient, and the preceding parts establish both properties for this matrix. If a residual becomes zero, stop: the solution has been reached and the displayed step quotient should not be evaluated as $0/0$.

For a convergence justification, let $x_{\ast }$ solve $A{x}_{\ast }=b$, let $e=x_{\ast }-x$, and let $r=Ae$. The quadratic energy error is $E=e^{T}Ae/2$. Moving by $\alpha r$, with $\alpha =(r^{T}r)/(r^{T}Ar)$, is exact line search for a positive-definite quadratic and givesWriting ${\lambda }_{\min },{\lambda }_{\max }>0$ for the extreme eigenvalues, one has $r^{T}Ar\le {\lambda }_{\max }{r}^{T}r$ and $r^{T}r=e^T{A}^{2}e\ge {\lambda }_{\min }{e}^{T}Ae$. ThereforeThe factor lies below one, so the energy tends to zero and positive definiteness forces $x\to x_{\ast }$. Here the bound is $E_{\mathrm{new}}\le (10/11)E$, since the eigenvalues are 1 and 11. This proves convergence for every initial vector, rather than merely checking one numerical iteration.

Set $y=P^{T}x$. Because $P$ is an orthogonal matrix, $x=Py$ and $x=0$ exactly when $y=0$. The preceding diagonalization givesConsequently $x^{T}Ax\ge 0$, with equality only when $x=0$. The matrix is a symmetric positive-definite matrix, not merely positive semidefinite. In the original coordinates the same identity iswhich also makes positivity transparent.

The supplied factors give $Q^{T}b=(0,2{)}^T$. Back substitution in $Rx=Q^{T}b$ yields $2y=2$ and $10x+4y=0$, soMultiplying by the original matrix gives $(1,1,1,1{)}^T$, so this least-squares minimizer has residual zero: this particular overdetermined system is exactly consistent. Full column rank still guarantees uniqueness of the minimizer.

At $d=1000$, gradient descent and full BFGS are practical candidates; a dense Newton method can also be feasible, but its extra work and indefinite curvature need attention. The number of examples $n$, data sparsity and conditioning still matter.

For dense $A$, a gradient evaluation costs $O(nd)$ arithmetic, with $O(d)$ working storage beyond the data and accumulation buffers. Gradient descent is inexpensive per iteration and, with an appropriate line search, follows a descent direction, but poor conditioning can make it slow.

Forming the Hessian matrix explicitly costs $O(n{d}^2)$ arithmetic, and a dense Newton method solve costs $O(d^3)$. A $1000$-square double-precision matrix uses about $8$ MB, so storage alone is manageable, but repeated factorization may be expensive. Because the Hessian matrix can be indefinite or singular, an unmodified Newton step need not decrease the loss; a modified Hessian, damping or a trust region is appropriate.

Full BFGS uses gradients and an $O(d^2)$ curvature approximation, avoiding explicit second derivatives. Its rank-two updates and direction calculation cost $O(d^2)$ per step and require roughly the same matrix storage. A positive-definite initial approximation with safeguarded curvature updates and a suitable line search preserves descent directions. For this scale it is often a good compromise, without guaranteeing a global minimum.^[ref]

At $d=10^6$, gradient descent remains a plausible large-scale choice; dense Newton method and full-memory BFGS are generally impractical. A dense $d\times d$ double-precision matrix alone contains $10^{12}$ entries, about $8$ TB. Dense factorization also has order $10^{18}$ arithmetic work. Full BFGS avoids factorization but retains quadratic storage and update work, so it does not solve that scaling problem.

Gradient descent uses linear working storage in $d$, though the cost of reading the data and evaluating the gradient is still $O(nd)$ for dense $A$. Sparsity or streaming can be important. Limited-memory BFGS, a distinct limited-memory variant, stores only a fixed number $m$ of recent update pairs and uses $O(md)$ storage and update work, making it a realistic alternative to full BFGS.^[ref]

The available Hessian matrix factorization also permits a matrix-free Hessian-vector product:It costs $O(nd)$ for dense data without forming $H$. A suitable iterative, safeguarded Newton method can exploit this structure, so the conclusion rules out the dense implementation rather than every possible second-order algorithm.

These methods seek a local solution and require convergence checks appropriate to nonconvex optimization. The formula does not even ensure a finite minimizer exists: with one positive coefficient and nonzero $\mathbf{a}$, the positive exponential loss approaches zero along an unbounded direction without attaining it. Dimension alone cannot supply a convergence guarantee.

For a differentiable scalar function of $n$ real variables, its derivative is the linear mapwhere the column gradient is $\nabla f=(\partial f/\partial x_1,\dots ,\partial f/\partial x_n{)}^T$. Its Hessian is the $n\times n$ matrix $H_f=({\partial }^{2}f/\partial x_i\partial x_j{)}_{i,j}$. For a $C^2$ function, equality of mixed partial derivatives makes this matrix symmetric.

Since $E^2=I$, the same matrix is both a left and a right inverse. $E^{-1}=E$.

The permutation matrix isRows three and four of the identity matrix have been interchanged, so left multiplication interchanges those rows of any matrix.

The first left multiplication swaps rows three and four. The final right multiplication swaps columns three and four, restoring the two diagonal ones to their diagonal positions while exchanging the earlier entries of the two rows. ThereforeThe two specified zeros at positions $(3,4)$ and $(4,3)$ are important. Conjugation by this permutation matrix preserves this restricted lower triangular pattern; it would not preserve lower triangularity for an arbitrary matrix with a nonzero $(4,3)$ entry.

Write $U=E{E}_5{E}_4{E}_3{E}_2{E}_{1}M$ and put $F_i=E{E}_{i}E$. Inserting $E^2=I$ between adjacent elimination matrices givesEach $E_i^{-1}$ has the restricted lower triangular pattern in part (b)(v). Also $F_i^{-1}=E{E}_i^{-1}E$, as direct multiplication verifies. Hence every $F_i^{-1}$ is lower triangular. Their product is lower triangular by the allowed closure property, proving the required Pivoted LU decomposition with $L=F_1^{-1}\cdots F_5^{-1}$.

Explicitly,The multipliers in rows three and four of $L$ are permuted along with the rows of $M$; simply keeping the unpermuted elimination multipliers would give the wrong product. This is the mechanism in conjugating elimination matrices across a row permutation.

Gaussian elimination gives the row operations $R_2\leftarrow R_2-3R_1$, $R_3\leftarrow R_3-R_1$, $R_4\leftarrow R_4-2R_1$, producing the lower three rows $(0,1,4,1)$, $(0,1,4,1)$, $(0,-3,2,-1)$. Subtract the new row two from row three and add three times row two to row four; the remaining nonzero row is $(0,0,14,2)$. Move it above the zero row and finish elimination to obtainThe pivot columns are one, two and three. Row operations preserve the linear dependence relations among columns, so the corresponding columns of the original matrix form a basis for the column space, which is the image of the linear map:In particular, the matrix rank is three. Columns of the reduced matrix itself would generally give a different subspace, so the original columns must be used.

Here a dense $100000$-by-$100000$ matrix has $10^{10}$ entries, about $80$ GB even with eight-byte storage per entry. Use limited-memory BFGS on the negative of the objective, or equivalently its positive-definite approximate-metric form of gradient ascent. It stores only a fixed small number $\ell$ of vector pairs, costing $O(\ell d)$ memory rather than $O(d^2)$.

Let $G(x)={\sum }_{i=1}^{100}{e}^{e^{-\Vert x-a_i{\Vert }^2}}$ and put $c_i(x)=e^{e^{-\Vert x-a_i{\Vert }^2}}{e}^{-\Vert x-a_i{\Vert }^2}$. Direct use of the chain rule givesIf $H_n$ denotes the positive-definite inverse-Hessian matrix approximation for $-G$, the iteration iswith a line search increasing $G$ and the usual safeguarded limited-memory updates. No exact Hessian matrix is required. With fixed memory and the usual bounded-metric and line search hypotheses, expect local linear convergence near a nondegenerate strict maximum; fixed-memory limited-memory BFGS does not generally inherit the superlinear theorem for full-memory BFGS.

Unlike the preceding objective, $G$ is not generally a concave function. Thus this is a method for finding a local maximum; comparison of different converged candidates or a global search is needed to certify a global maximum for arbitrary centres. The symbolic problem supplies no assumption that would justify an unconditional global-convergence claim.^[ref]