Minimum-variance spectral contrast orthogonal to a reference shape ($\mathbf{a}=C^{-1}{\mathbf{s}}_{\perp }/({\mathbf{s}}_{\perp }^T{C}^{-1}{\mathbf{s}}_{\perp })$)

A minimum-variance spectral contrast orthogonal to a reference shape estimates a desired signal amplitude while exactly rejecting one fixed nuisance shape. Let $\mathbf{z}=\theta \mathbf{s}+\beta \mathbf{x}+\mathbf{ϵ}$, with known shapes $\mathbf{s},\mathbf{x}$ and a positive-definite noise covariance matrix $C$. Choose $\hat{\theta }={\mathbf{a}}^T\mathbf{z}$ to minimize ${\mathbf{a}}^{T}C\mathbf{a}$, subject toThe constraints retain unit signal response and reject the reference amplitude, as in projection of shared-scale uncertainty onto a spectral contrast. The quadratic variance includes the full noise covariance.^[ref]

Remove the reference component in the inverse-covariance geometry:For nonzero $\mathbf{x}$ and $D>0$, the minimizing contrast and its variance areFor a short proof, every other feasible contrast is $\mathbf{a}+\mathbf{v}$ with ${\mathbf{v}}^T\mathbf{s}={\mathbf{v}}^T\mathbf{x}=0$. Since $C\mathbf{a}$ is a linear combination of those shapes, the cross term vanishes, leaving the added nonnegative variance ${\mathbf{v}}^{T}C\mathbf{v}$. This is a constrained best linear unbiased estimator.

In an original three-channel example, take $\mathbf{s}=(1,0,0{)}^T$, $\mathbf{x}=(1,1,1{)}^T$ and $C=\mathrm{diag}(1,4,1)$. The optimum is $\mathbf{a}=(1,-1/5,-4/5{)}^T$, with variance $1.8$. Equal subtraction of the two reference channels, $(1,-1/2,-1/2{)}^T$, has variance $2.25$. The quieter third channel receives more of the subtraction.

If the desired and reference shapes are proportional, $D=0$ and no unit-response rejecting contrast exists. The noise penalty from nearly parallel signal and reference shapes already becomes large as $D$ approaches zero. This construction assumes fixed known shapes and covariance; estimating them from the same noisy spectra requires additional uncertainty terms.

The noise penalty from nearly parallel signal and reference shapes is the precision lost when a spectral estimator must reject a nuisance pattern resembling its desired signal. For nonzero shapes $\mathbf{s},\mathbf{x}$ and positive-definite noise covariance $C$, define their inverse-covariance correlationThe Cauchy-Schwarz inequality gives $|r|\le 1$. The minimum-variance spectral contrast orthogonal to a reference shape has varianceIf the nuisance amplitude were known and subtracted without uncertainty, the corresponding optimum variance would be $V_{\mathrm{known}}=1/({\mathbf{s}}^T{C}^{-1}\mathbf{s})$. Exact nuisance rejection therefore multiplies variance by $1/(1-r^2)$ and standard uncertainty by $1/\sqrt{1-r^2}$.

The relevant angle is the noise-whitened angle between spectral signal and nuisance shapes: it is measured after a whitening transformation, not generally from the raw plotted spectra. A channel with large noise contributes little to separating the shapes even if their plotted curves differ there.

For an original example, let $C={\sigma }^{2}I$, $\mathbf{x}=(1,0{)}^T$ and $\mathbf{s}=(1,0.1{)}^T$. Rejecting the reference forces the contrast to $(0,10{)}^T$, with variance $100{\sigma }^2$. With known nuisance amplitude, its variance would be ${\sigma }^2/1.01$. The rejection penalty is therefore $101$ in variance, about $10.05$ in standard uncertainty. Only the weak second-channel component distinguishes signal from nuisance.

The primary stimulated-Raman experiment reported systematic spectral noise above its calculated shot-noise limit and described pure-solvent subtraction of a common pattern.^[ref] A reference can remove that shared pattern while also making an overlapping desired signal hard to estimate. Retain full covariance in the variance calculation.^[ref] At exact proportionality, no unit-response rejecting contrast exists; a low-noise result obtained by suppressing the signal as well is not a valid amplitude estimate.

The noise-whitened angle between spectral signal and nuisance shapes measures their separation in units of measurement noise rather than raw plotting units. For nonzero shapes $\mathbf{s},\mathbf{x}$ and a positive-definite covariance $C$, choose a square whitening transformation $W$ with $WC{W}^T=I$. The angle isIt is the ordinary Euclidean angle after whitening. Both parallel and antiparallel shapes are indistinguishable up to an unknown nuisance amplitude; the precision penalty depends on ${\sin }^2\varphi$, as in noise penalty from nearly parallel signal and reference shapes.

For an original example, take raw shapes $\mathbf{s}=(1,1{)}^T$, $\mathbf{x}=(1,-1{)}^T$ and $C=\mathrm{diag}(1,100)$. The plotted shapes are orthogonal, but whitening gives $(1,0.1{)}^T$ and $(1,-0.1{)}^T$. Thus $\cos \varphi =0.99/1.01\approx 0.98020$ and $\varphi \approx 11.42^{\circ }$. The second channel, which separates their signs, is noisy. Rejecting the nuisance multiplies the optimum variance by $1/{\sin }^2\varphi =25.5025$, or standard uncertainty by $5.05$.

The invariance of noise-whitened spectral angles to the whitening factor follows from $W^{T}W=C^{-1}$. Different valid square whitening factors can rotate the whitened coordinates without changing the angle. Likewise an invertible change of measurement units transforms both shapes and covariance, leaving the inverse-covariance expression unchanged. Rescaling only the displayed curves while leaving their noise model fixed is a different operation.

The covariance must describe the actual measured channels and shared errors, using the full propagation rule.^[ref] If $C$ is singular, first restrict to its physically allowed range; an unrestricted inverse whitening factor does not exist.

The primary stimulated-Raman experiment described systematic spectral noise and subtraction of a pure-solvent reference.^[ref] The angle above assesses which desired patterns remain distinguishable after such nuisance rejection under a stated covariance. It does not establish that an estimated reference shape or noise model is itself accurate.

Invariance of noise-whitened spectral angles to the whitening factor means that any two complete whitening transformations for the same positive-definite covariance give the same angles between transformed shapes. Suppose square invertible matrices satisfy $W_{i}C{W}_i^T=I$. Put $Q=W_2{W}_1^{-1}$. Since $C=W_1^{-1}{W}_1^{-T}$,Thus $Q$ is orthogonal. It may rotate or reflect the whitened coordinates, but preserves their inner products, norms and angles. The noise-whitened angle between spectral signal and nuisance shapes is consequently a property of the shapes and covariance, not a particular whitening algorithm.

For an original example, chooseDirect multiplication gives $W_{1}C{W}_1^T=I$. With $\mathbf{s}=(2,1{)}^T$ and $\mathbf{x}=(2,2{)}^T$, the whitened shapes are $(1,0{)}^T$ and $(1,1{)}^T$, meeting at $45^{\circ }$. Using $W_2=Q{W}_1$ instead gives $(0,1{)}^T$ and $(-1,1{)}^T$, again at $45^{\circ }$, although the coordinate components differ.

SciPy describes covariance whitening as transforming variables to unit covariance and illustrates it with a precision-matrix construction.^[ref] The matrix proof above establishes the invariance independently of an implementation's chosen factor.

The same principle requires covariance-consistent rescaling of whitened spectral coordinates. If raw measurement units change by an invertible matrix $D$, transform both shapes by $D$ and covariance by $C^{\prime }=DC{D}^T$; then $W^{\prime }=W_1{D}^{-1}$ reproduces the same whitened shapes. Changing a plotted channel's scale without transforming its noise model is not this invariant operation.

The claim assumes a full whitening of a positive-definite covariance. Truncating modes can remove information and change angles, while a singular covariance requires restriction to its supported range. Uncertainty in an estimated covariance is also separate from this exact algebraic invariance.

Covariance-consistent rescaling of whitened spectral coordinates changes measurement units or mixes coordinates while transforming their noise model at the same time. If an invertible matrix $D$ maps a spectral vector to ${\mathbf{y}}^{\prime }=D\mathbf{y}$, thenThe covariance follows directly from the expectation of the transformed error outer product. For positive-definite $C$,Thus the inverse-covariance inner product, its norms and the resulting whitened angle are invariant. Invariance of noise-whitened spectral angles to the whitening factor establishes the related freedom to rotate the fully whitened coordinates.

For an original example, let $C=\mathrm{diag}(1,9)$, $\mathbf{s}=(1,1{)}^T$ and $\mathbf{x}=(1,-1{)}^T$. Their noise-whitened angle has cosine $0.8$, or angle about $36.87^{\circ }$. Converting only the first raw coordinate's numerical units by a factor of 1000 gives $D=\mathrm{diag}(1000,1)$ and $C^{\prime }=\mathrm{diag}(10^6,9)$. Using that transformed covariance keeps the cosine exactly $0.8$.

If the shapes are rescaled but the old covariance is mistakenly retained, the calculated cosine becomes approximately $0.99999978$, suggesting an angle only about $0.0382^{\circ }$. A unit conversion would then appear to destroy the separation between signal and nuisance. That apparent change comes from inconsistent uncertainty accounting.

For any full whitening factor $W$, the transformed factor $W^{\prime }=W{D}^{-1}$ satisfies $W^{\prime }{C}^{\prime }(W^{\prime }{)}^T=I$ and gives $W^{\prime }{\mathbf{s}}^{\prime }=W\mathbf{s}$. The pinned SciPy documentation describes whitening and demonstrates a near-identity covariance after transformation.^[ref] Measurement propagation likewise retains all covariance terms under a linear transformation.^[ref]

The invertibility condition matters. Noninvertible spectral compression versus a coordinate change can discard information and need not preserve these angles. A transformation estimated from the same noisy observations also brings estimation uncertainty beyond the fixed-matrix identity proved here.

Noninvertible spectral compression versus a coordinate change distinguishes discarding spectral coordinates from expressing the same measurements in different units. The covariance-consistent rescaling of whitened spectral coordinates article assumes an invertible transformation. A compression $\mathbf{z}=B\mathbf{y}$ with fewer independent rows than input coordinates instead has covariance $C_z=BC{B}^T$ and may lose information even when this covariance is propagated correctly.^[ref]

For positive-definite $C$ and a known signal shape $\mathbf{s}$, compare the squared noise-weighted signal sizesassuming $B$ has independent rows. Put $\mathbf{t}=C^{-1/2}\mathbf{s}$ and $M=B{C}^{1/2}$. ThenThe matrix $P$ is an orthogonal projector. This proves that compression cannot increase this signal measure under the stated covariance model. The condition for preserving a spectral signal under linear compression is that its whitened shape lie entirely in the retained subspace.

For an original example, let $C=I_2$, desired shape $\mathbf{s}=(1,1{)}^T$ and nuisance shape $\mathbf{x}=(1,-1{)}^T$. Keeping only the first channel, $B=(1,0)$, reduces $J$ from two to one and makes both shapes the same scalar. An unknown nuisance amplitude can then imitate the desired signal exactly.

Summing instead, $B=(1,1)$, gives desired response two, nuisance response zero and noise variance two. It retains $J_B=2$ for this particular signal while rejecting that nuisance. Thus a noninvertible map can preserve a chosen estimation task, but cannot in general be reversed to recover every original spectral component.

The primary Raman apparatus records array spectra before subsequent spectral processing.^[ref] Choose compression against the scientific signal and nuisance model, and propagate the full covariance. Correct uncertainty accounting does not restore a spectral distinction removed by the map.

The condition for preserving a spectral signal under linear compression specifies when compression retains the covariance-weighted signal measure of a known shape. Let the mean be $\theta \mathbf{s}$, with fixed positive-definite noise covariance $C$, and compress by a full-row-rank matrix $B$. The measures in noninvertible spectral compression versus a coordinate change obeyThus the rows of the compression must retain the noise-weighted signal direction, not necessarily the unweighted plotted shape.

For a short proof, whiten the shape as $\mathbf{t}=C^{-1/2}\mathbf{s}$ and put $M=B{C}^{1/2}$. The parent's orthogonal projection formula givesEquality holds exactly when $\mathbf{t}$ lies in the row space of $M$. Multiplying that subspace relation by $C^{-1/2}$ yields the stated condition. Linear covariance propagation gives the compressed covariance $BC{B}^T$.^[ref]

One useful construction is the scalar matched filter statistic $z={\mathbf{s}}^T{C}^{-1}\mathbf{y}$. Its mean is $\theta J$ and variance $J$, so $z/J$ has variance $1/J$, the original optimal linear amplitude precision. In the Gaussian known-covariance model, this statistic also retains the likelihood's dependence on that single amplitude.

For an original example, choose $C=\mathrm{diag}(1,4)$ and $\mathbf{s}=(1,2{)}^T$. Then $J=2$ and the retaining row is $B=(1,1/2)$. Its signal response is two and noise variance two, giving $J_B=4/2=2$. An unweighted sum $B=(1,1)$ instead has response three and variance five, giving $J_B=9/5=1.8$: ten percent of this signal measure is lost.

Signal-subspace preservation for several spectral amplitudes requires retaining the corresponding family of weighted mean directions when all those amplitudes are to be preserved. Unknown nuisance shapes can also change the relevant estimation problem. A compression designed for one known shape does not automatically preserve every spectral question or uncertainty component.

Signal-subspace preservation for several spectral amplitudes requires a compression to retain every noise-weighted mean direction relevant to those amplitudes. Let $\mathbf{y}$ have mean $A\mathbf{\theta }$, fixed positive-definite noise covariance $C$, and a full-column-rank shape matrix $A$. Compress it as $B\mathbf{y}$, with $B$ full row rank. The original and compressed amplitude-information matrices areFor Gaussian noise these are Fisher information matrices. Without that distributional assumption, they still describe the corresponding known-covariance linear estimation problem.

Whiten the shapes as $T=C^{-1/2}A$ and put $M=B{C}^{1/2}$. The orthogonal projection onto the row space of $M$ is $P=M^T(M{M}^T{)}^{-1}M$, givingThis difference is positive semidefinite. It vanishes exactly when every column of $T$ lies in that row space, equivalentlyThus preserving one selected shape does not establish preservation of the whole signal subspace. Linear covariance propagation supplies $BC{B}^T$ for the compressed observations.^[ref]

For an original example, take $C=\mathrm{diag}(1,4,9)$ and shape columns $(1,2,0{)}^T$ and $(0,0,3{)}^T$. Then $J=\mathrm{diag}(2,1)$. Choosinggives $BA=\mathrm{diag}(2,1)$ and $BC{B}^T=\mathrm{diag}(2,1)$, hence $J_B=J$. Keeping only the first compressed row retains information2 for the first amplitude but destroys sensitivity to the second. An unweighted first row $(1,1,0)$ would instead reduce the first information entry to1.8.

At least $p$ independent compressed directions are needed to preserve $p$ independent shape amplitudes. One construction is $B=A^T{C}^{-1}$, whose compressed signal has mean $J\mathbf{\theta }$ and covariance $J$. When $J$ is invertible, $J^{-1}B\mathbf{y}$ has mean $\mathbf{\theta }$ and covariance $J^{-1}$. In the fixed-covariance Gaussian model, expanding the likelihood function shows that its parameter dependence is retained through $A^T{C}^{-1}\mathbf{y}$.

Unknown nuisance shapes, changing covariance or additional spectral parameters can alter the subspace that needs preservation. The proof concerns the specified family of mean shapes and fixed noise model, rather than guaranteeing that one compression preserves every future spectral question.