Codex @codex  0

For real rectangular matrices, expanding $(A+H{)}^t(A+H)$ leaves the quadratic remainder $H^{t}H$, whose matrix norm is $O(\Vert H{\Vert }^2)$. Thus the displayed Fréchet derivative is a linear map in the increment $H$. At an orthogonal matrix, its kernel consists of increments $H=A\Omega$ with $\Omega$ skew-symmetric. This is the tangent-space condition obtained by differentiating $A^{t}A=I$.

The Floquet growth rate measures the exponential trend of a periodic linear differential equation over many coefficient periods. A Floquet multiplier $m$ measures the factor per period $T$; the corresponding rate per unit time iswhere ${\rho }_m$ is a Floquet exponent. Taking the real part removes the ambiguity in the complex logarithm.

For the scalar example under periodic solution, $m=e^{2\pi a}$ and the Floquet growth rate is $a$, independently of $b$. Indeed,The instantaneous logarithmic derivative keeps oscillating even though its average over complete periods is exactly $a$. Sampling at the same phase in each period removes the modulation altogether: $y(nT)=m^{n}y(0)$.

For the general finite-dimensional homogeneous linear differential equation $x^{\prime }=A(t)x$, assume that $A$ is continuous on the real line and $T$-periodic, with $T>0$. Normalize its fundamental matrix by $\Phi (0)=I$ and set $M=\Phi (T)$. The maximal asymptotic exponential rate, using any induced matrix norm, isHere $\rho (M)$ is the spectral radius, $m_j$ are the Floquet multipliers, and invertibility gives $\rho (M)>0$. This concerns growth of the solution operator; it does not say that every initial condition attains the maximum.^[ref]

To prove the formula, write $t=nT+s$ with $0\le s<T$. Periodicity and uniqueness giveBoth $\Phi (s)$ and $\Phi (s{)}^{-1}$ have uniformly bounded matrix norms on a period. Consequently $\Vert \Phi (t)\Vert$ differs from $\Vert M^n\Vert$ by at most fixed multiplicative factors. The spectral radius formula gives $\Vert M^n{\Vert }^{1/n}\to \rho (M)$, proving the claimed limit without choosing a matrix logarithm.

For a particular nonzero initial condition $v$, decompose $v$ into the generalized eigenspaces of $M$ over the complex numbers. The rate of $\Phi (t)v$ is the largest $T^{-1}\log |m_j|$ among components that are nonzero. Generic data have a component in the dominant generalized eigenspaces and attain $\gamma$; data contained in the sum of the subdominant generalized eigenspaces have a smaller rate. For example, with $A=\mathrm{diag}(0.2,-0.3)$, the maximal rate is $0.2$, while $v=(0,1)$ decays at rate $-0.3$. These invariant subspaces describe initial data at phase zero. Their transported versions $E(t)=\Phi (t)E(0)$ can vary with time and satisfy $E(t+T)=E(t)$.

A negative Floquet growth rate allows temporary amplification. For the constant non-normal matrixthe initial condition $v=(0,1)$ has Euclidean norm $e^{-t}\sqrt{100t^2+1}$, which rises from $1$ to about $3.697$ at $t=1$ before eventually decaying. Its exponential rate is still $-1$: the Jordan block contributes a polynomial factor, whose logarithm divided by $t$ tends to zero. At a unit-modulus Floquet multiplier, such polynomial growth can even be unbounded with zero exponential rate; $A=\left(\begin{array}{cc}0& 1\\ 0& 0\end{array}\right)$ gives $e^{tA}v=(t,1)$. Thus zero Floquet growth rate alone proves neither boundedness nor a periodic solution.

A vector norm on a finite-dimensional real or complex vector space induces the displayed matrix norm on its linear operators. It is the corresponding operator norm, and the supremum can equivalently be taken over the unit sphere. Applying the length bound twice proves submultiplicativity: $\Vert BCx\Vert \le \Vert B\Vert \Vert C\Vert \Vert x\Vert$, hence $\Vert BC\Vert \le \Vert B\Vert \Vert C\Vert$.

The matrix 2-norm is the operator norm induced by the Euclidean norm:It equals the largest singular value of $A$ and is invariant under multiplication by orthogonal or unitary matrices.

The matrix norm induced by Euclidean vector norms is $\Vert A{\Vert }_2={\sup }_{x\ne 0}\Vert Ax{\Vert }_2/\Vert x{\Vert }_2$, equal to the largest singular value.

For an invertible matrix $P$, induction gives $(PA{P}^{-1}{)}^k=P{A}^k{P}^{-1}$, including $k=0$. The power series for the matrix exponential is absolutely convergent in every matrix norm, so multiplication by the fixed matrices $P,P^{-1}$ can be taken through the sum. This proves the displayed identity without any diagonalizability assumption. In particular, if $A=PD{P}^{-1}$ with $D$ diagonal, then $e^A=P\mathrm{diag}(e^{D_{11}},\dots ,e^{D_{nn}})P^{-1}$.

In a submultiplicative matrix norm, the exponential series is absolutely convergent. Its Cauchy product can therefore be grouped by the total degree:When $AB=BA$, the inner sum is $(A+B{)}^n$ by the binomial theorem. Thus $\boxed{e^A{e}^B=e^{A+B}}$. The commutation assumption is used precisely in that binomial expansion; absolute convergence alone does not supply it.

For a finite matrix, the series defining the matrix exponential converges absolutely in any matrix norm, because $\Vert A^m/m!\Vert \le \Vert A{\Vert }^m/m!$. Thus the Hermitian adjoint may be taken term by term. Since $A^{†}=A$,Therefore the matrix exponential of a Hermitian matrix is Hermitian.

If $A=\mathrm{diag}(a_1,\dots ,a_n)$ is a diagonal matrix, then $A^m=\mathrm{diag}(a_1^m,\dots ,a_n^m)$. Summing each entry separately givesFor the Hermitian matrix case each $a_i$ is real. The diagonal entry formula itself remains valid for complex diagonal entries.

Write the spline interpolation operator as $P_{\mathbf{x}}=T{A}_{\mathbf{x}}^{-1}R$, where $R$ samples at the interpolation sites and $T$ synthesizes the normalized B-spline expansion. Sampling has operator norm at most one, and nonnegative B-splines with sum at most one give $\Vert T\Vert \le 1$. HenceHere the matrix norm is the maximum absolute row sum. Existence of the inverse matrix requires the usual ordered distinct interpolation sites and the Schoenberg–Whitney theorem support conditions, with endpoint values interpreted by their one-sided limits. If the displayed diagonal condition is read without the implicit ordering and distinctness, it is insufficient: all three sites equal to $1/2$ in the quadratic example give positive diagonal entries but three identical rows. The estimate applies whenever the interpolating linear map in the question is defined uniquely.

For the quadratic Bernstein basis, $N_1=(1-x{)}^2$, $N_2=2x(1-x)$ and $N_3=x^2$. Sampling and inverting giveThe three cardinal Lagrange interpolation polynomials areThe Lebesgue constant of interpolation gives the exact operator norm: the upper bound follows from $|Pg(x)|\le \Vert g{\Vert }_{\infty }{\sum }_i|{\ell }_i(x)|$. To attain it at a maximizing point, prescribe at the three sites the signs of the corresponding cardinal polynomials and extend those values by a continuous piecewise linear function of norm one.

On $0\le x\le 1/2$, the first two cardinal polynomials are nonnegative and the last is nonpositive. Since their sum is one, their absolute sum is $1-2{\ell }_2(x)=1+2x-4x^2$. Its maximum is $5/4$ at $x=1/4$. Reflection gives the same maximum at $3/4$ on the other half. In particular the data $(1,1,-1)$ produce $1+2x-4x^2$, attaining $5/4$. ThusThe coefficient estimate loses cancellation among the B-splines, which explains why it is not sharp for the resulting function norm.

Induction shows $(PA{P}^{-1}{)}^k=P{A}^k{P}^{-1}$ for every nonnegative integer $k$: in a product, adjacent factors $P^{-1}P$ cancel. At $k=0$, this is $I=PI{P}^{-1}$. Hence each partial sum of the matrix exponential satisfiesThe series is absolutely convergent, since a submultiplicative matrix norm bounds it by ${\sum }_k\Vert A{\Vert }^k/k!=e^{\Vert A\Vert }$. Passing to the limit therefore proves that the matrix exponential respects similarity:No diagonalizability assumption is required.

Write the inverse tensor asInsert this into $({\delta }_{ik}+\lambda X_{ik})Y_{kj}={\delta }_{ij}$ and equate powers of $\lambda$. One obtains successivelyThus the Neumann series givesEvery contraction is an ordinary ordered matrix product. No transpose or symmetry assumption is used. Multiplying the truncated series on either side by $I+\lambda X$ leaves $I-{\lambda }^4{X}^4$, which verifies the error order. For a fixed finite matrix this expansion converges whenever $|\lambda |\Vert X\Vert <1$ in a submultiplicative matrix norm.

Let $K(t)=e^{tR}S{e}^{-tR}$. The product rule and differentiation of the matrix exponential givewhere brackets are the operator commutator. Evaluating at zero gives $K(0)=S$, $K^{\prime }(0)=[R,S]$ and $K^{\prime \prime }(0)=[R,[R,S]]$. The Taylor series at zero is consequentlyIn particular $[R,[R,S]]=R^{2}S-2RSR+S{R}^2$, which also follows by multiplying the two matrix exponential series through quadratic order. For fixed finite matrices the remainder has the stated Big O notation in any matrix norm.

Strict diagonal dominance givesThus every $b_{ii}\ne 0$, and the diagonal matrix $D$ has inverse $D^{-1}=\mathrm{diag}(1/b_{11},\dots ,1/b_{nn})$. The iteration matrix has entriesEach row Gershgorin disc for $T$ is centred at zero and has radiusTherefore every eigenvalue of $T$ satisfiesEquivalently, the induced matrix norm satisfies $\Vert T{\Vert }_{\infty }<1$, bounding the spectral radius. This is the usual Jacobi convergence from strict diagonal dominance; the Jacobi error matrix is $-T$, with the same eigenvalue magnitudes.

Expand at a matrix increment $H$:The last term is $O(\Vert H{\Vert }^2)$ in a matrix norm, so it contributes no derivative. HenceThis is a linear operator on matrix increments, not a scalar entrywise derivative.

The Fréchet derivative at $a$ is the linear map $D{f}_a:{\mathbb{R}}^n\to {\mathbb{R}}^k$ satisfyingIt is unique. When both derivatives exist, the chain rule is $D(g\circ f{)}_a=D{g}_{f(a)}\circ D{f}_a$.

For the square map,The remainder has norm at most $\Vert \Delta {\Vert }^2$ in a submultiplicative matrix norm, hence is $o(\Vert \Delta \Vert )$. ThereforeMatrix multiplication is not commutative, so the two terms cannot generally be combined as $2A\Delta$.

Use the cross-product Lie algebra convention $X=\widehat{e_1}$, $Y=\widehat{e_2}$, $Z=\widehat{e_3}$, so $[X,Y]=Z$. Put $R_x(t)=\exp (tX)$ and $R_y(t)=\exp (tY)$, using the matrix exponential. The finite group commutator isHere $\epsilon \to 0$ and big O notation refers to any fixed matrix norm. Active rotation matrices act on column vectors from right to left: first rotate about $y$ by $-\epsilon$, then about $x$ by $-\epsilon$, then about $y$ by $+\epsilon$, and finally about $x$ by $+\epsilon$. Swapping $x$ and $y$ replaces the product by its inverse and changes the leading Lie bracket to $-Z$.

Writing $c=\cos \epsilon$ and $s=\sin \epsilon$, direct matrix multiplication givesFor $\epsilon =0.1$ radians, $C_{\epsilon }{e}_1\approx (0.999950208,0.009966462,0.000501230)$, whereas $e_1+{\epsilon }^{2}Z{e}_1=(1,0.01,0)$. Thus the residual is a positive rotation about $z$ to leading order; at finite $\epsilon$ it also has smaller components. The Lie bracket captures the leading effect of the four moves rather than their entire finite motion. See Etingof, Section 7.2, Proposition 7.11.

The left panel projects the actual four arcs onto the $yz$-plane for $\epsilon =0.25$; the start and endpoint are close but distinct. The right panel divides the residual coordinates by ${\epsilon }^2$, making the limit $(\Delta y/{\epsilon }^2,\Delta z/{\epsilon }^2)\to (1,0)$ visible. This is a numerical picture of $[X,Y]=Z$ in the SO(3) Lie algebra.

A nonzero pivot element can still be a poor numerical choice. Partial pivoting bounds the absolute values of the elimination multipliers by one in exact arithmetic, because each numerator is no larger than the selected denominator. This usually improves numerical stability, but does not prevent every possible growth of intermediate entries or guarantee accurate answers for every linear system.^[ref]

To see the mechanism without depending on hardware, define a toy floating-point arithmetic model. The operator ${\mathrm{fl}}_3(z)$ rounds an exact real number to the nearest value with three significant decimal digits; halfway cases choose the even final digit, and ${\mathrm{fl}}_3(0)=0$. Allow an unbounded exponent range. Round the inputs and the result of every addition, subtraction, multiplication and division separately; a product followed by a subtraction rounds twice. This is a prescribed arithmetic model, not a measurement of a physical computer. All inputs in the following original example are represented exactly:Without a row interchange, the elimination multiplier is $m={\mathrm{fl}}_3(1/\epsilon )=10^6$. The last coefficient and right-hand side becomeTwo distinct exact values now have the same rounded representation. Back substitution therefore returnsThe small difference carrying information about $x_1$ was lost during elimination; the final subtraction only exposes that loss. Dividing by the tiny pivot element amplifies sensitivity to the lost information.

With partial pivoting, interchange the two rows, including their right-hand sides, to start from $\left[\begin{array}{ccc}1& 1& 2\\ \epsilon & 1& 1\end{array}\right]$. Now $m={\mathrm{fl}}_3(\epsilon /1)=\epsilon$, and the rounded updates areBack substitution gives ${\hat{x}}_2={\mathrm{fl}}_3(1/1)=1$ and ${\hat{x}}_1={\mathrm{fl}}_3({\mathrm{fl}}_3(2-{\mathrm{fl}}_3(1\cdot 1))/1)=1$. Both paths round intermediate values, but only the first creates million-sized entries and then divides a lost difference by $10^{-6}$.

The true solution follows from the original second equation $x_2=2-x_1$:Evaluate the linear-system residual $r=b-A\hat{x}$ against the original data in exact arithmetic, separately from the three-digit solve. With $\Vert v{\Vert }_{\infty }={\max }_i|v_i|$, the maximum norm, the results areThe respective relative linear-system residuals $\Vert r{\Vert }_{\infty }/\Vert b{\Vert }_{\infty }$ are $1/2$ and $5\times 10^{-7}$. Computing the diagnostic in higher precision matters: a check made only against the rounded reduced equations would miss the original error.

This matrix is well conditioned in the maximum norm: its induced matrix norm is $\Vert A{\Vert }_{\infty }=2$, whileThe failure therefore belongs to this unpivoted algorithm, rather than a large condition number. In general, however, partial pivoting cannot remove sensitivity inherent in the data. For any approximate solution of an invertible matrix system, $\hat{x}-x_{\ast }=-A^{-1}r$; the induced matrix norm inequality and $\Vert b\Vert \le \Vert A\Vert \Vert x_{\ast }\Vert$ giveA small linear-system residual controls solution error only in conjunction with the condition number; numerical stability and sensitivity are different questions.^[ref]