Rank deficiency from deterministic spectral upsampling
ID: rank-deficiency-from-deterministic-spectral-upsampling
Rank deficiency from deterministic spectral upsampling occurs because a deterministic map cannot manufacture independent measurements. For and input covariance matrix ,When the output has more channels than the input, its covariance matrix is necessarily singular. Covariance of count-conserving rebinned Raman channels gives the conserved-count example and its full propagated uncertainty.[ref]
Using that parent's two-to-three channel map, the output always satisfies . If the input readings are and , the rebinned vector is . Recovering the original readings as and shows that the third displayed number is not a third independent observation.
For an original fitting example, suppose both input means equal an unknown and their independent measurement-error variances are known to be . Fitting in the original channels gives , with variance . A correct generalized least squares fit after rebinning gives the same result and uncertainty. It must use the singular covariance on its supported range, for example through the Moore-Penrose inverse, together with the exact linear constraints.
The range restriction in singular-covariance spectral fitting matters if a proposed output model fails those constraints. A residual along cannot arise from this rebinning model at all. Merely inserting a pseudoinverse into a quadratic objective would assign zero penalty to that impossible direction. Either enforce the range condition or supply a physically justified additional noise model; adding a small diagonal term solely to make the matrix invertible changes the statistical assumptions.
The motivating Raman apparatus measures detector-array spectra before any subsequent numerical grid change.[ref]
Femtosecond Broadband Stimulated Raman: A New Approach for High-Performance Vibrational Spectroscopy by David W. McCamant and others. Instrumentation, array detection and spectral calibration; Data Collection, equation 4
Working on the original grid often avoids this singular fitting problem while retaining the actual measurement information. New to topics? Read the docs here!