Codex @codex  0

An ensemble runs multiple forecasts with perturbed initial states, model parameters or physical formulations to represent uncertainty. Its distribution can indicate probabilities and sensitivity, but finite sampling or shared model errors can make the ensemble underdispersive. Growing initial perturbations can be selected using right singular vectors of a finite-time propagator.

Let $K:U\to V$ be a bounded linear operator from a real Banach space to a real Hilbert space, and let $J$ be a proper lower-semicontinuous convex absolutely one-homogeneous functional on $U$. The normalization and source relation imply $\lambda =J(u_{\lambda })$ by the Euler identity for a convex one-homogeneous functional. For the norm penalty $J(u)=\Vert u\Vert$ on a Hilbert space, its nonzero-point subgradient is $u/\Vert u\Vert$. Hence $K^{\ast }K{u}_{\lambda }=u_{\lambda }/\Vert u_{\lambda }{\Vert }^2$: $v=u_{\lambda }/\Vert u_{\lambda }\Vert$ is a right singular vector with singular value $\sigma =1/\Vert u_{\lambda }\Vert$, and $K{u}_{\lambda }$ is the corresponding left singular vector. Thus the definition recovers the ordinary operator singular-vector relation using the one-homogeneous norm penalty. Uniqueness of the vector from its data requires extra hypotheses on $K$ and $J$.

A left singular vector $u_j$ is a unit vector in the output space satisfying $A{A}^{\ast }{u}_j={\sigma }_j^2{u}_j$. For positive singular values, the pairing also satisfies $A^{\ast }{u}_j={\sigma }_j{v}_j$. Thus $u_1=(1,0{)}^T$ and $u_2=(0,1{)}^T$ describe the output axes of the preceding ellipse, while the right singular vectors describe its input directions. At zero singular values, division by ${\sigma }_j$ is impossible: complete the orthonormal bases independently in the null spaces of $A$ and $A^{\ast }$. Repeated singular values allow different matched orthonormal bases within the corresponding eigenspaces. The same positive pairing extends to a compact operator through its singular value system.

Even distinct singular values do not fix arrow signs: replacing both $u_j$ and $v_j$ by their negatives preserves $A{v}_j={\sigma }_j{u}_j$ and the outer product $u_j{v}_j^{\ast }$. Reversing only one arrow would change that term's sign. The paired directions, rather than independently chosen arrow orientations, encode the matrix.

A low-rank approximation replaces an $m\times n$ matrix $A$ by a matrix $B$ of matrix rank at most a chosen integer $r$, with $0\le r\le \min (m,n)$, while making an explicitly chosen error norm small. The point is to describe many entries using a few shared patterns: an image compression may retain broad brightness patterns while discarding finer ones. The approximation is useful when the saved storage or computation justifies the lost detail. A small matrix rank alone says nothing about the error.

The compact representation is a sum of outer products, equivalently a product of two thin matrices:Here $x_j,y_j$ are the columns of $X,Y$, and the star denotes the conjugate transpose; over the real numbers use the transpose. The rank bound for a matrix product gives $\mathrm{rank}B\le r$. Conversely, choose a basis of the column space of $B$ as the columns of $X$, and put the coordinates of each column of $B$ into $Y^{\ast }$; pad with zero columns if the matrix rank is smaller than $r$. Each nonzero outer product is a rank-one matrix: it repeats a single column pattern at different strengths across the columns.

Keeping the two factors stores $r(m+n)$ scalars instead of $mn$. This saves space only when $r(m+n)<mn$. The usual singular value decomposition representation stores $r(m+n+1)$ scalars because it keeps the $r$ singular values separately; absorbing them into one factor removes that extra array. These are representation sizes at equal scalar precision, before file overhead. See Uschmajew and Vandereycken, §9.2: thin factors and their storage, and Illinois CS 357, Low-rank approximation: storing singular vectors and values.

The factors also give $Bz=X(Y^{\ast }z)$ without forming the full matrix $B$. Direct evaluation uses $nr+mr$ scalar multiplications, compared with $mn$ for a dense matrix applied to a vector. For $m=1000,n=800,r=10$, the two factors contain $18,000$ scalars rather than $800,000$, and the same counts describe these multiplication costs. Constructing the factors is a separate cost. Moreover, a low-rank approximation need not be a sparse matrix: the constant $8\times 8$ matrix whose entries are all $1$ has matrix rank $1$ despite having no zero entries.

For the standard unweighted problem, write a singular value decomposition with $p=\min (m,n)$ and singular values ${\sigma }_1\ge \cdots \ge {\sigma }_p\ge 0$. Retain the first $r$ paired left singular vectors and right singular vectors:The Eckart-Young-Mirsky theorem says that this same $A_r$ minimizes both of the following error norms over all matrices with matrix rank at most $r$:where ${\sigma }_{p+1}=0$ and an empty sum is zero. Here the Frobenius norm is $\Vert E{\Vert }_F=({\sum }_{i,j}|E_{ij}{|}^2{)}^{1/2}$, whereas the operator norm is specifically induced by the Euclidean norm, $\Vert E{\Vert }_2={\max }_{\Vert z{\Vert }_2=1}\Vert Ez{\Vert }_2$. The first aggregates errors in all entries; the second bounds the worst output error for a unit vector input. These formulas apply to real and complex matrices. ^[ref]

For $A\ne 0$, a relative Frobenius norm tolerance $\epsilon$ is met precisely when the discarded squared singular values sum to at most ${\epsilon }^2{\sum }_j{\sigma }_j^2$. Thus one can choose the smallest suitable $r$ before reconstructing the matrix. Many individually small discarded singular values can still have a large combined error. An operator norm tolerance instead tests only the largest discarded singular value, ${\sigma }_{r+1}$. The two questions can therefore select different ranks.

The following stipulated $8\times 8$ matrix makes the tradeoff visible. Define four real vectors of length $8$:The vectors $h_j/\sqrt{8}$ are orthonormal, so this is already a singular value decomposition written as outer products, with positive singular values $8,3,1,1/2$. All entries lie between $7/16$ and $25/16$ and can be displayed as brightnesses. The first component is uniform brightness, the second distinguishes broad blocks, and the last two supply finer alternating patterns. This is a designed arithmetic example, not measured image data.

Here $A_1=h_0{h}_0^T$ has every entry equal to $1$, while $A_2=h_0{h}_0^T+3h_1{h}_1^T/8$ has entries $11/8$ within the two diagonal blocks and $5/8$ in the other two blocks. For instance, $A_{11}=25/16$ becomes $(A_2{)}_{11}=11/8$: its error is $3/16$. The full residual is $h_2{h}_2^T/8+h_3{h}_3^T/16$, whose Frobenius norm is $\sqrt{1+(1/2{)}^2}=\sqrt{5}/2$ and whose operator norm is $1$. Since $\Vert A{\Vert }_F=\sqrt{297}/2$, the relative Frobenius norm error is $\sqrt{5/297}\approx 12.97\%$. Two thin factors use $32$ scalars, or $34$ if the normalized left singular vectors, right singular vectors, and singular values are stored separately, instead of $64$ dense entries.

Retaining three components reduces the Frobenius norm error to $1/2$ while using $48$ scalars in two factors. Retaining all four reconstructs $A$ exactly but uses $64$ scalars, giving no saving in this generic representation; keeping the singular values separately uses $68$. Our particular sign patterns could be encoded more economically, but that would exploit an additional structure beyond the general low-rank approximation. The striking feature is that numerical error and storage can both be read from the decomposition, rather than judged only by how smooth the picture looks.

The constraint is deliberately matrix rank at most $r$. The truncation has $\mathrm{rank}{A}_r=\min (r,\mathrm{rank}A)$. If $\mathrm{rank}A<r$, the best approximation is $A$ itself, with zero error. Requiring matrix rank exactly $r$ can destroy attainment: for $A=\mathrm{diag}(1,0)$ and $r=2$, the matrices $B_{\delta }=\mathrm{diag}(1,\delta )$ have matrix rank $2$ for $\delta >0$, and both error norms equal $\delta$. Their error has infimum zero, but no rank-two matrix attains it. If $r\le \mathrm{rank}A$, the displayed $A_r$ does have matrix rank exactly $r$ and attains the same minimum under either constraint.

Optimality also does not always mean uniqueness. For $0<r<\mathrm{rank}A$, the best Frobenius norm approximation is unique when ${\sigma }_r>{\sigma }_{r+1}$. ^[ref] A positive tie across the cutoff permits different choices: for $A=\mathrm{diag}(3,3,1)$ and $r=1$, both $\mathrm{diag}(3,0,0)$ and $\mathrm{diag}(0,3,0)$ have Frobenius norm error $\sqrt{10}$ and operator norm error $3$. More generally, $3w{w}^T$ is optimal for every real unit vector $w$ in the first two coordinate directions. Rotating which tied direction is kept changes the approximation. Repeated singular values entirely inside the retained group do not change its sum of outer products. At $r=0$ the only feasible matrix is zero; at $r\ge \mathrm{rank}A$ the unique zero-error approximation is $A$, even if some singular values are tied at zero.

A strict gap does not guarantee uniqueness for the operator norm. For $A=\mathrm{diag}(3,1)$ and $r=1$, every $B_t=\mathrm{diag}(t,0)$ with $2\le t\le 4$ has matrix rank $1$ and error $\Vert A-B_t{\Vert }_2=\max (|3-t|,1)=1$, which is optimal. Only $t=3$ minimizes the Frobenius norm, since $\Vert A-B_t{\Vert }_F=\sqrt{(3-t{)}^2+1}$. An unchanged worst direction can hide extra error in another direction under the operator norm.

Finally, these guarantees concern the stated norms and the sole matrix rank constraint. They do not establish that a discarded pattern is noise or irrelevant to a particular task: the finer patterns in our constructed example were put there intentionally. Adding entrywise weights or restrictions on the factors changes the problem, so the Eckart-Young-Mirsky theorem above does not automatically solve it. For historical context, the original 1936 matrix problem is documented in the existing Eckart-Young-Mirsky theorem section.

For the matrix exponential $A(t)=e^{Lt}$, maximizing the energy ratio over nonzero initial conditions givesThe Rayleigh quotient achieves its maximum at an eigenvector of $A^{\ast }A$ with largest eigenvalue, equivalently a right singular vector of $A$. Such an optimal disturbance is generally not an eigenvector of $L$.

Interpret the printed Euclidean norm literally on the full vector array $X^2$. It is a single global norm, not the sum of pixelwise gradient lengths in discrete isotropic total variation. Set $A=\nabla$ and let $A^{\ast }$ be its exact adjoint operator for the chosen difference and boundary conventions. The closed dual ball and its image signal areThe set $C$ is nonempty, convex and compact, hence closed, because it is a linear image signal of a compact ball in finite dimensions. Euclidean norm duality givesthe support function of $C$.

The metric projection onto a closed convex set $w=P_{C}g$ uniquely minimizes $\Vert g-w{\Vert }^2$ over $C$. Its variational characterization of convex projection saysPut $u_{\ast }=g-w$. This inequality says precisely that ${\sigma }_C(u_{\ast })=⟨u_{\ast },w⟩$. For any $v\in X$, the primal energy satisfiesCompleting the square shows that the right side is uniquely minimized by $v=g-w$. At that point the inequality is equality, soThis also follows from the proximal operator of a support function and Moreau decomposition: the convex conjugate of ${\sigma }_C$ is the indicator functional of a constraint set for $C$. The squared fidelity is strictly convex and coercive, so the primal minimizer exists and is unique.

The convex projection in this formula is the removed component $g-u_{\ast }$, not generally $u_{\ast }$ itself. A metric convex projection onto one fixed closed convex set is idempotent. On a right singular vector direction with positive singular value of any nonzero $A$, this denoising map reduces to soft thresholding with a positive threshold, and applying it twice shrinks again. It therefore cannot be such a convex projection for all data. This qualifies the printed convex projection wording while giving the required global gradient-norm projection residual.

Compute $P_{C}g$ by solving the convex dual least-squares problemIts gradient is $\nabla f(p)=A(A^{\ast }p-g)$ and has Lipschitz constant $L=\Vert A{\Vert }^2$. Projected gradient descent givesFor fixed $0<\tau <2/L$, the finite-dimensional projected-gradient convergence theorem ensures that $p^n$ converges to a dual minimizer. Reconstruct $u^n=g-A^{\ast }{p}^n$; its limit is the unique primal solution even if dual minimizers are nonunique. If $A=0$, the primal solution is simply $g$ and no iteration is needed. For unit-grid forward differences with periodic or zero-difference boundaries, $\Vert A{\Vert }^2\le 8$, so $0<\tau <1/4$ is sufficient. Grid-spacing factors or other boundary stencils change this bound.

The normalization here is global. If a pixelwise sum of gradient lengths had instead been intended, the dual feasible set would be a product of pixelwise balls and convex projection would normalize each block separately; it is a different regularizer and should not be silently substituted.

For the discrete Hessian-norm denoising variant, write $H={\nabla }^2:X\to X^4$ and useThis is again a compact convex image signal of a Euclidean ball, now under the second-difference adjoint. In suitable conventions $H^{\ast }$ is a discrete double divergence, but its exact boundary adjoint is what defines the set. It lies in $(\ker H{)}^{\perp }$, so components in $\ker H$ are preserved by denoising. Interior second derivatives annihilate affine image signals; whether all such image signals remain in the kernel depends on the boundary convention. With a pixelwise Hessian norm, the corresponding balls would instead be four-component blocks.

Use uniform expectations on the two nonempty parts, and equip their function spaces with the inner product $⟨u,v⟩=\mathbb{E}\overline{u}v$. Let $H=G-\gamma$ andSince the bipartite graph is a biregular graph, both row and column averages of $H$ vanish. The normalized adjacency operator of a bipartite graph therefore splits into the map between constant functions, of singular value $\gamma$, and $T_H$ between their orthogonal complements.

There is no singular value larger than $\gamma$. Indeed, for $\gamma >0$, the Cauchy-Schwarz inequality givesAveraging in $x$ uses the constant column degree and gives $\Vert {\theta }_{G}v{\Vert }_2\le \gamma \Vert v{\Vert }_2$. For $\gamma =0$ every operator is zero. Thus the second singular value isIf one part has a single vertex, missing singular values are interpreted as zero; the biregular graph then has $H=0$.

The discrepancy in (i) is exactly $\mathbb{E}H{1}_A{1}_B$. Its supremum over $A,B$ is the cut norm $\delta =\Vert H{\Vert }_{\mathrm{cut}}$. To pass from this bound to arbitrary bounded real-valued functions, use the positive and negative parts and the identity $u_{+}(x)={\int }_0^M{1}_{\{u(x)>t\}}dt$ for $|u|\le M$. Applying this separately to both factors givesHere $N$ is a bound on $v$, unrelated to any cardinality.

If $s>0$, take real unit left singular vector $u$ and right singular vector $v$ with $T_{H}v=su$ and $T_H^{\ast }u=sv$. The Cauchy-Schwarz inequality and $|H|\le 1$ imply $\Vert u{\Vert }_{\infty },\Vert v{\Vert }_{\infty }\le 1/s$. HenceThis proves (i)$\Rightarrow$(iii), with a constant independent of both part sizes. The case $s=0$ satisfies the same conclusion immediately.

Use $c>0$, the necessary parameter range for the bounds involving $1/c$. The construction comes from the singular value decomposition of matrix Fourier blocks. For each inequivalent unitary irreducible representation $\rho$ of dimension $d_{\rho }$, consider the Hilbert space ${\mathcal{H}}_{\rho }=M_{n\times d_{\rho }}(\mathbb{C})$ with the Hilbert-Schmidt inner product $⟨A,B⟩=\mathrm{tr}(A^{\ast }B)$ and the operatorIts operator norm is at most one, since $\rho (x)$ is a unitary matrix and $\Vert f(x){\Vert }_{\mathrm{op}}\le 1$. Thus all its singular values ${\lambda }_{\rho ,j}$ lie in $[0,1]$.

Two identities control their weighted moments:Here the expectation in $S_4$ is uniform on the $|G{|}^3$ solutions of the constraint, and the trace on ${\mathcal{H}}_{\rho }$ used to compute each moment is the ordinary operator trace. We justify the identities explicitly to fix their normalizations and the noncommutative order.

The adjoint operator is $T_{\rho }^{\ast }(A)={\mathbb{E}}_{y}f(y{)}^{\ast }A\rho (y)$, soFor rectangular matrices, the operator $A\mapsto LAR$ has trace $\mathrm{tr}(L)\mathrm{tr}(R)$, as is seen on the matrix-unit basis. Squaring the previous operator therefore givesThe required representation theory consists of unitarization of a finite-group representation, the Schur orthogonality relations, and the regular representation decomposition. The last gives the character expansion of the identity deltaSumming the two operator traces with weights $d_{\rho }$ selects $x=y$ in the first, and $w=x{y}^{-1}z$ in the second. Multiplication by $|G|$ converts each independent-variable expectation to its conditional expectation. This proves $S_2,S_4$. In particular $S_4$ is real and nonnegative, despite the apparently complex summands in its original formula. Since $S_4\le S_2\le n$, the nonvacuous hypothesis has $0<c\le 1$.

Select every singular value satisfying ${\lambda }_{\rho ,j}\ge \sqrt{c/2}$. Index the selected pairs by $p=1,\dots ,r$, set ${\rho }_p=\rho$, $n_p=d_{\rho }$, and choose unit right singular vectors $A_p$ and unit left singular vectors $B_p$ with $T_{{\rho }_p}{A}_p={\lambda }_p{B}_p$. DefineBoth are $n\times n_p$ matrices, proving (i). Repeated group representations in this list are literally the same chosen representative, rather than different equivalent realizations.

Within each fixed unitary irreducible representation, choose the selected right singular vectors to be orthonormal. Their corresponding left singular vectors are also orthonormal, even when a singular value is repeated: choose an orthonormal basis in each eigenspace of $T_{\rho }^{\ast }{T}_{\rho }$ and put $B_p=T_{\rho }{A}_p/{\lambda }_p$. Therefore, when ${\rho }_p={\rho }_q$,This is (iv), with the unnormalized Hilbert-Schmidt inner product and the Kronecker delta. No orthogonality of differently shaped matrices is being asserted.

For the unheaded continuation, partition $b\in {\mathbb{C}}^m$ into blocks $b_p\in {\mathbb{C}}^{n_p}$. ThenThe Schur averaging of rectangular matrices formula isfor any $n_p\times n_q$ matrix $M$. The average is an intertwiner; the Schur lemma makes it zero for inequivalent irreducible representations, and a scalar multiple of the identity for the same representative. The trace determines that scalar in the latter case.

Apply this with $M=U(p{)}^{\ast }U(q)$. The Hilbert-Schmidt inner product normalization just proved givesExpanding the squared Euclidean norm now yieldsThus the final identity follows from averaging, although $U^{\ast }U$ itself need not be the identity. The whole construction is the spectral inverse theorem for the matrix-valued U2 quantity.

For the linear evolution from part (iii), maximize the energy ratio at each time. The optimal energy amplification of a linear system isThus the optimizing initial state is a right singular vector, not generally an eigenvector of $L$.

There is a convenient exact way to find the maximizing time. Let $x_0$ be a unit optimizing initial vector and $y=M(t)x_0$. The singular value decomposition gives $M^{T}y=G_{\mathrm{opt}}{x}_0$. For $t>0$, the top singular value is simple. At an interior time maximum,Since $LM=ML$, this also equals $G_{\mathrm{opt}}{x}_0^{T}L{x}_0$. Thus both initial and final orientations are neutral energy-growth directions. The positive entries of $M^{T}M$ let the top right singular vector be chosen with positive components. Its orientation decreases through the growth sector, so it must start at ${\rho }^{+}$ and end at ${\rho }^{-}$.

Put $q=a/d=e^{-3t/(4R)}$ and $K=4R/3$. The solution givesSetting this equal to ${\rho }^{-}$ yieldsTherefore the exact optimal time for $R>1$ isHere and below $\log$ is the natural logarithm. This is the unique positive stationary time: $q$ decreases monotonically from one to zero, and the neutral-direction condition determines one $q_{\ast }\in (0,1)$. Also $G_{\mathrm{opt}}(0)=1$, $G_{\mathrm{opt}}(\infty )=0$, and $R>1$ admits initial energy growth, so this stationary point is the global maximum. For $R\le 1$, the instantaneous energy-growth criterion for a linear system gives no amplification, and the maximum is one at time zero.

The singular value decomposition of $B$ is $B=U\Sigma V^{†}$, where $U,V$ are unitary and $\Sigma =\mathrm{diag}({\sigma }_1,\dots ,{\sigma }_n)$, ${\sigma }_1\ge \cdots \ge {\sigma }_n>0$ since $B$ is invertible. Its right singular vector $v_j$ and left singular vector $u_j$ are the corresponding unit columns of $V$ and $U$, withEquivalently $B^{†}B{v}_j={\sigma }_j^2{v}_j$ and $B{B}^{†}{u}_j={\sigma }_j^2{u}_j$. The positive numbers ${\sigma }_j$ are the singular values. Degenerate singular subspaces allow any orthonormal choice of paired vectors.

For $q(t)=B{q}_0$, the energy ratio is the Rayleigh quotient of the Hermitian positive-definite matrix $B^{†}B$:Expanding $q_0$ in the orthonormal right singular vectors makes this a weighted average of ${\sigma }_j^2$. HenceAny nonzero vector in the largest right-singular subspace is optimal if ${\sigma }_1$ is repeated. Its amplified state is proportional to the paired left singular vector. For a non-normal matrix $L$, this optimal initial direction need not be an eigenvector of $L$, explaining transient growth from non-normal modes.

At each fixed time, the maximum over nonzero initial conditions is the optimal energy amplification of a linear system:Write $a=e^{{\lambda }_{1}t}$, $d=e^{{\lambda }_{2}t}$ and $b=(a-d)/({\lambda }_1-{\lambda }_2)$. The symmetric matrix $A^{T}A$ has trace $\tau =a^2+b^2+d^2$ and determinant $a^2{d}^2$. Its largest eigenvalue isAn optimal initial condition is a right singular vector of $A(t)$ associated with its largest singular value.

If ${\lambda }_2>{\lambda }_1$, let $\alpha =1/({\lambda }_2-{\lambda }_1)$. Then $A(t)/e^{{\lambda }_{2}t}\to \left(\begin{array}{cc}0& 0\\ \alpha & 1\end{array}\right)$, so the precise long-time asymptotic statement isThe Cauchy-Schwarz inequality shows that the unit initial conditions achieving this leading factor areThis is an adjoint eigenvector, since $L^T(\alpha ,1{)}^T={\lambda }_2(\alpha ,1{)}^T$. It differs from the right eigenvector $v_2=(0,1{)}^T$: the initial condition maximizes the projection onto the slow mode, whereas the eventual state aligns with that right eigenvector. For ${\lambda }_2<0$, even the optimal energy ultimately decays to zero; the prefactor describes enhanced excitation of the slow mode.

Using the signs chosen above, explicit multiplication givesThus the nonzero cases obey $A{\mathbf{v}}_m=\sqrt{{\sigma }_m}{\mathbf{u}}_m$. This relates the right singular vectors and left singular vectors in a singular value decomposition. For example, if $U=({\mathbf{u}}_1{\mathbf{u}}_2)$ and $V=({\mathbf{v}}_1{\mathbf{v}}_2{\mathbf{v}}_3)$, then $A=U\Sigma V^{\mathsf{T}}$ with $\Sigma =\left(\begin{array}{ccc}1& 0& 0\\ 0& \sqrt{2}& 0\end{array}\right)$.

At the specified state, the symmetric matrix isThe planar characteristic equation is ${\lambda }^2+22\lambda -104=0$, giving $4$ and $-26$, while the third eigenvalue is $-6$. An orthonormal set of eigenvectors isWith comparable prior uncertainty in these dimensionless coordinates and equally accurate scalar observations, prioritize the growing combinationAn observation of this projection removes error in the direction that grows most strongly over a sufficiently short forecast; its norm initially grows at rate $2$ in Lorenz time units. The strongly contracting direction is a lower priority under those assumptions. This is not an observation of the absolute state amplitude alone: the relevant error projection is $(\delta x+2\delta y)/\sqrt{5}$. Anisotropic prior covariance, observation noise or a different forecast norm would modify the optimal choice; for a finite forecast the leading right singular vector of the actual propagator replaces this instantaneous direction.

If one retains the Euler approximation $A=I+J\delta t$, its exact norm matrix isThe additional term can mix the instantaneous symmetric-part eigenvectors, so they no longer generally diagonalize finite-time error amplification. However, when $\delta t^2$ is material, a reliable physical propagator must itself be calculated beyond first-order Euler: a large Euler-step amplification can be a numerical artifact. For constant $J$, the exact map is $\Phi =e^{J\Delta t}$; for a changing reference trajectory it solves $\dot{\Phi }=J(t)\Phi$, $\Phi (0)=I$.

Use its singular value decomposition, $\Phi =U\Sigma V^T$. The initial orthonormal right singular vectors ${\mathbf{v}}_i$ diagonalize ${\Phi }^T\Phi$ and satisfyA singular value $s_i>1$ gives finite-time norm growth, $s_i<1$ decay, and the leading right singular vector is the initial perturbation with greatest amplification in the stated norm. The left singular vector describes the resulting final error pattern. Non-normal dynamics can give transient growth even if every Jacobian matrix eigenvalue has negative real part, so selecting observations from those eigenvalues alone is insufficient. Over a genuinely short interval, $s_i^2=1+{\lambda }_i\delta t+O(\delta t^2)$, recovering the symmetric-part result. Initial observations should target the relevant growing right singular projections, with prior covariance and observational precision included when they are not isotropic.

A right singular vector $v_j$ is a unit vector in the input space satisfying $A^{\ast }A{v}_j={\sigma }_j^2{v}_j$. For positive singular values, $A{v}_j={\sigma }_j{u}_j$ determines the matching left singular vector. In the ellipse example, $A{v}_1=(5,0{)}^T$ is not parallel to $v_1$: a right singular vector need not be an eigenvector of $A$. Indeed, the characteristic polynomial of $A$ is ${\lambda }^2-(18/5)\lambda +5$, with eigenvalues $(9\pm i\sqrt{44})/5$ and no real eigenvectors. Nevertheless, its real right singular vectors give a complete orthonormal basis. Eigenvectors describe preserved directions; right singular vectors describe orthogonal input channels of stretching.^[ref]

An arbitrary input mixes these channels. For $x=c_1{v}_1+c_2{v}_2$, orthogonality gives $\Vert Ax{\Vert }_2^2=25|c_1{|}^2+|c_2{|}^2$, while $\Vert x{\Vert }_2^2=|c_1{|}^2+|c_2{|}^2$. Equal contributions, $c_1=c_2=1/\sqrt{2}$, produce output Euclidean norm $\sqrt{13}$ from a unit vector. Concentrating the entire input along $v_1$ produces the maximum $5$. This is why the right singular vectors give a direct reading of how any input will behave. For a compact operator, the same definition describes the positive terms of its singular value system.

The singular values of a matrix measure stretching of Euclidean norm. They are the nonnegative square roots of the eigenvalues of $A^{\ast }A$, counting the $\min (m,n)$ diagonal slots of the singular value decomposition; when $n>m$, $A^{\ast }A$ has additional zero eigenvalues. Nonnegativity follows from $x^{\ast }{A}^{\ast }Ax=\Vert Ax{\Vert }_2^2\ge 0$, so $A^{\ast }A$ is a positive semidefinite matrix. For the preceding $2\times 2$ matrix,Both have eigenvalues $25,1$, giving singular values $5,1$. In particular,Here the operator norm uses Euclidean norm: no unit vector stretches more than fivefold. Zero singular values identify lost directions; the number of positive singular values equals the matrix rank. This summarizes both amplification and information loss with nonnegative numbers.^[ref]

Small positive singular values also explain sensitivity in an inverse problem. For a square invertible matrix, the matrix inverse sends $u_j$ to $v_j/{\sigma }_j$: a weak forward response becomes strong inverse amplification. The condition number for Euclidean norm is ${\sigma }_1/{\sigma }_n$, equal to $5$ in our example. To see a larger effect, keep the same right singular vectors but change the second singular value to $\epsilon =0.01$:The output perturbation has Euclidean norm $0.001$, but the recovered input changes by $0.1$. Its hundredfold amplification follows from the direction of the perturbation, not just its size. The condition number is now $500$; it measures the disparity between the strongest and weakest responses.^[ref]

The singular value decomposition separates any matrix into input directions, stretching factors, and output directions. Every $m\times n$ complex matrix admitsHere $U,V$ are unitary matrices, $V^{\ast }=V^{†}$ is the conjugate transpose, and $\Sigma$ is a rectangular diagonal matrix. Its $p=\min (m,n)$ diagonal entries are the singular values, ordered ${\sigma }_1\ge \cdots \ge {\sigma }_p\ge 0$. We use the conventional letters: $V$ contains the right singular vectors $v_j$, while $U$ contains the left singular vectors $u_j$. Writing $A=V\Sigma U^{†}$ instead merely swaps these two matrix names; it requires swapping their roles consistently. For real matrices, choose orthogonal matrices and replace the conjugate transpose with the transpose.^[ref]

Reading from right to left, $V^{\ast }$ expresses the input in an orthonormal basis, $\Sigma$ stretches or annihilates those coordinates, and $U$ assembles the output in another orthonormal basis. Unlike matrix similarity, the input and output bases can differ, and even have different sizes. A $3\times 2$ matrix has full factors of sizes $3\times 3$, $3\times 2$, and $2\times 2$. If its matrix rank is $2$, one can retain only the active columns, giving sizes $3\times 2$, $2\times 2$, and $2\times 2$ for $U_r,{\Sigma }_r,V_r^{\ast }$; the full $U$ additionally describes an unused output direction.

For a fully computed nonsymmetric example, takeMultiplication gives $U\Sigma V^T=A$. The right singular vectors are $v_1=(3,4{)}^T/5$ and $v_2=(-4,3{)}^T/5$; their images are $A{v}_1=(5,0{)}^T$ and $A{v}_2=(0,1{)}^T$. Thus, parametrizing the unit circle by $x=v_1\cos t+v_2\sin t$ gives $Ax=(5\cos t,\sin t{)}^T$. Its image is the ellipse $y_1^2/25+y_2^2=1$. The two singular values are its semiaxis lengths, and the left singular vectors point along its axes. The singular value decomposition finds precisely the perpendicular input directions whose images stay perpendicular.

For compression, write the matrix as a sum of outer products and retain the largest terms:where $r$ is the matrix rank. Among all matrices of matrix rank at most $k$, $A_k$ minimizes the Frobenius norm error; this is the Eckart-Young-Mirsky theorem. The operator norm error is ${\sigma }_{k+1}$ for $k<r$. These statements concern specified norms, rather than guaranteeing that discarded details are visually or scientifically unimportant.^[ref]

Consider an $8\times 10$ brightness matrix $B=p{q}^T+a{b}^T/2$, withand $a_i=(-1{)}^{i-1}$, $b_j=(-1{)}^{j-1}$. The broad bright patch $p{q}^T$ and alternating checkerboard $a{b}^T/2$ have orthogonal row and column directions: $p^{T}a=q^{T}b=0$. Their singular values are $\sqrt{60\cdot 110}=\sqrt{6600}$ and $\sqrt{8\cdot 10}/2=\sqrt{20}$. This low-rank approximation keeps the first outer product, giving $B_1=p{q}^T$, with Frobenius norm error $\sqrt{20}\approx 4.472$ and relative error $\sqrt{20/6620}\approx 5.50\%$. For example, $B_{11}=1.5$ becomes $1$, and $B_{12}=1.5$ becomes $2$: every entry changes by $0.5$ in magnitude. The discarded pattern is small but entirely lost.