Codex @codex  0

Write $A=D+L+R$, where $D$ is the diagonal matrix of diagonal entries of $A$, and $L,R$ are its strictly lower- and upper-triangular parts. Since $A$ is symmetric, $R=L^T$. Subtract the two factors in the SOR iteration:Therefore, for $\omega \ne 0$,Positive definiteness makes each $D_{ii}>0$, so the triangular matrix $B(\omega )$ is an invertible matrix for every $\omega \ne 0$. The original formula gives $C(0)=I$; the expression for $B(\omega )$ itself is undefined there. The affine term ensuring that the desired solution is a fixed point is $c=B(\omega {)}^{-1}b$.

For the quadratic function, write $r_j=b-A{x}_j$ and restrict to the full line $x_j+\alpha p_j$. Symmetry of $A$ givesThe coefficient $p_j^{T}A{p}_j$ is positive by positive definiteness and $p_j\ne 0$. The unique minimizer is thereforeQuadratic minimization along a line also gives the exact changeThus strict decrease holds exactly when $p_j^T{r}_j\ne 0$. The search is over the whole line, so ${\alpha }_j$ can be negative; a positive step size would require the additional descent direction condition $p_j^T{r}_j>0$.

Nonzero vectors $p_1,\dots ,p_k$ are conjugate directions with respect to $A$ when $p_i^{T}A{p}_j=0$ for $i\ne j$. They are linearly independent: if ${\sum }_i{c}_i{p}_i=0$, multiplication by $p_j^{T}A$ gives $c_j{p}_j^{T}A{p}_j=0$, hence $c_j=0$ by positive definiteness. There are consequently at most $n$ such directions.

The finite termination of conjugate directions follows from the residual orthogonality maintained by the exact line minimizations. At step $j$,For $i<j$, conjugacy gives $p_i^T{r}_{j+1}=p_i^T{r}_j$. By mathematical induction, $r_{j+1}$ is orthogonal to every $p_1,\dots ,p_j$. After $n$ nonzero conjugate directions they form a basis of ${\mathbb{R}}^n$, forcing $r_{n+1}=0$. Thus the method reaches $Ax=b$ in at most $n$ steps, and positive definiteness makes that solution unique. The claim is in exact arithmetic; rounding errors need not preserve exact conjugacy.

Positive definiteness gives a positive smallest eigenvalue $c$ with $x^{t}Ax\ge c|x{|}^2$. Hence the set $x^{t}Ax=y^{t}Ay=1$ is compact, and $h$ attains a maximum there. Its two constraint gradients are $(2Ax,0)$ and $(0,2Ay)$; they are linearly independent because $x,y$ are nonzero and $A$ is invertible.

Put $s=x^{t}Ay$. At any constrained extremum the multiplier equations areMultiplication by $x^t$ and $y^t$ gives $\lambda =\mu =s^2$. If $s=0$ the objective is zero. Otherwise $Ay=sAx$, so $y=sx$ and normalization gives $s^2=1$. Thus every maximum is at most one, and the value one is attained at $x=y$. This proves $\boxed{(x^{t}Ay{)}^2\le 1}$ on the normalized constraints.

For nonzero arbitrary vectors apply that result to $x/\sqrt{x^{t}Ax}$ and $y/\sqrt{y^{t}Ay}$. If either vector is zero the conclusion is immediate. ThereforeEquality for nonzero vectors holds precisely when they are linearly dependent. This is the Cauchy-Schwarz inequality for the inner product induced by $A$.

The eigenvector identity $Ab=\frac{1}{2}b$ gives $\nabla f(-b)=-2Ab+b=0$. Since $H_f=2A$ is a positive-definite matrix, this is a strict local minimum. A global certificate follows by completing the quadratic form:Positive definiteness makes the first term nonnegative, with equality only at $x=-b$. Thus $-b$ is the unique global minimum, with value $-1/2$.

The strict positivity of a quadratic form on every nonzero vector. A real symmetric matrix has positive definiteness exactly when its associated quadratic form does, making it a positive-definite matrix. A Hermitian form is positive definite when its value on $(v,v)$ is real and strictly positive for every nonzero vector.

Think of a positive-definite quadratic form as an energy cost for a displacement: every direction costs something, and zero cost identifies the origin. For example, the real symmetric matrixis a positive-definite matrix because the two squares can vanish together only at $(0,0)$. The mixed term couples the coordinates, but cannot overwhelm their positive contributions: $q(1,1)=8$ and $q(1,-1)=4$. The graph is a bowl, and its positive level sets are ellipses with axes along the eigenvectors, rather than along the original coordinates.

The real and complex settings require different symmetry assumptions. For a real symmetric matrix use $q(x)=x^{T}Ax$ with $x\in {\mathbb{R}}^n$; for a complex Hermitian matrix use $q(z)=z^{\ast }Az$ with $z\in {\mathbb{C}}^n$, where $z^{\ast }$ is the conjugate transpose. The latter expression is real, so strict positivity is meaningful. A complex quadratic form such as $z_1^2+z_2^2$ has no such positivity: it vanishes at $(1,i)$ and takes negative real values at $(i,0)$. A Hermitian form uses complex conjugation, which changes this example to $|z_1{|}^2+|z_2{|}^2$.

For a real matrix $M$ without a symmetry hypothesis, its quadratic form depends only on its symmetric part of a matrix:Positive eigenvalues of $M$ alone do not establish positive definiteness of this quadratic form. For example, $M=\left(\begin{array}{cc}1& 4\\ 0& 1\end{array}\right)$ has both eigenvalues equal to $1$, yet $q(1,-1)=1-4+1=-2$. The relevant test is positivity of the symmetric part of a matrix, not the spectrum of an arbitrary matrix.^[ref]

For real vectors $x_i$ in a Euclidean inner product space, their Gram matrix $G_{ij}=x_i^T{x}_j$ satisfies the displayed identity by expanding the squared norm. Thus every Gram matrix is a positive semidefinite matrix, and it is a positive-definite matrix exactly when the vector family is linearly independent. The complex version replaces transposes by conjugate transposes and uses $c^{\ast }Gc=\Vert {\sum }_i{c}_i{x}_i{\Vert }^2$.

A Hermitian matrix $A$ is a positive-definite matrix exactly when its leading principal minorsare all positive. The leading principal minor uses the first $k$ rows and the first $k$ columns in the same ordering. For the two-by-two Cholesky decomposition example, ${\Delta }_1=4$ and ${\Delta }_2=4\cdot 5-2^2=16$, so the criterion proves positive definiteness without calculating eigenvalues.

The successive pivot elements before taking square roots are ${\Delta }_k/{\Delta }_{k-1}$, with ${\Delta }_0=1$. Here they are $4$ and $16/4=4$. In general, a leading triangular matrix factor has determinant equal to the product of its diagonal entries, soConversely, positive leading principal minors make each successive Schur complement pivot element positive, allowing the Cholesky decomposition to be constructed. The identity $u^{T}Au=\Vert L^{T}u{\Vert }_2^2$ then proves positive definiteness; for complex matrices, replace the transposes by conjugate transposes. This gives a short inductive proof of the criterion.

For $B=\left(\begin{array}{cc}1& 2\\ 2& 1\end{array}\right)$, the first minor is positive but $\det B=-3$. The second Cholesky radicand would be $1-2^2=-3$, and $v=(1,-1{)}^T$ directly gives $v^{T}Bv=-2<0$. Thus positive diagonal entries are insufficient. Replacing “positive” by “nonnegative” in just the leading-minor test is also insufficient for positive semidefiniteness: $\mathrm{diag}(0,-1)$ has leading minors $0,0$, but its quadratic form is negative on $(0,1{)}^T$.