Spectral-density conversion between wavelength and Raman shift

Spectral-density conversion between wavelength and Raman shift preserves the integrated physical quantity while changing its horizontal coordinate. Fix the excitation wavelength ${\lambda }_0$, and let $s$ be the Raman shift. Using numerical vacuum wavelengths in nanometres and shifts in $\mathrm{c}{\mathrm{m}}^{-1}$,Let $D_{\lambda }$ denote detected counts per nanometre and $D_s$ counts per unit Raman shift. Conservation of counts under a change of variables givesThe conversion factor carries the corresponding density-unit change. The same argument applies to calibrated power densities, provided both sides describe the same quantity.

For an original narrow bin centred at $600\mathrm{nm}$, a count density of $1200$ per nanometre becomes $43.2$ per $\mathrm{c}{\mathrm{m}}^{-1}$. A one-nanometre bin there spans approximately $27.78\mathrm{c}{\mathrm{m}}^{-1}$, so its integrated count remains about $1200$. At $800\mathrm{nm}$, the same wavelength density becomes $76.8$ per $\mathrm{c}{\mathrm{m}}^{-1}$. Relabeling bin positions without adjusting the density would conceal this coordinate dependence.

For finite bins, use their exact transformed edges and preserve each bin's integral. Raman spectral rebinning with conserved integrated counts distinguishes this operation from interpolating peak heights onto a new grid. Instrument response correction is another step; the Jacobian alone does not calibrate detector sensitivity or remove background.

The primary FSRS apparatus calibrated Raman shifts using known solvent lines and dispersed the probe spectrum onto an array.^[ref] Its dimensionless Raman pump-on pump-off gain ratio is a different observable: a ratio of matched spectral densities can cancel their common coordinate factor. It should not be treated as a count density merely because it is plotted against Raman shift.

Raman spectral rebinning with conserved integrated counts redistributes finite-bin measurements onto a new spectral grid while preserving their summed quantity over the same covered range. It is different from relabeling bin centres or interpolating peak heights. The parent spectral-density conversion between wavelength and Raman shift derives the continuous Jacobian; finite bins require a stated assumption about unresolved variation inside each bin.

For an original piecewise-constant wavelength-density model, let old bin $I_i$ have width $\Delta {\lambda }_i$ and count $N_i$. Invert the new Raman-shift edges to wavelength intervals $J_j$. Define the overlap matrixWhen the new bins partition the old covered range, ${\sum }_j{A}_{ji}=1$, hence ${\sum }_j{N}_j^{\prime }={\sum }_i{N}_i$. Clipping the new range instead excludes the corresponding counts and must be reported.

For original old wavelength bins $[600,602]$ and $[602,604]\mathrm{nm}$ with counts $100$ and $300$, choose new edges corresponding to wavelengths $600$, $601$, $603$ and $604\mathrm{nm}$. ThenThe total remains $400$. A different within-bin model can distribute that total differently; the original measurements do not resolve every sub-bin feature.

Covariance of count-conserving rebinned Raman channels transforms as $C^{\prime }=AC{A}^T$ for this fixed linear operation. Fractional rebinned estimates are therefore not automatically independent Poisson photon counts. Splitting one measured bin among neighbours creates shared uncertainty; finer output pixels do not create additional spectral resolution.

The primary FSRS instrument records a dispersed spectrum on an array and calibrates its Raman-shift axis.^[ref] The overlap algorithm above is an original illustrative reconstruction, not a claim about that instrument's published processing. Apply radiometric response and background corrections consistently with the quantity whose integral is being conserved.

The covariance of count-conserving rebinned Raman channels records the shared noise created when one measured channel contributes to several output channels. For a fixed deterministic rebinning matrix $A$ and an input count vector $\mathbf{N}$ with covariance matrix $C$, the output is ${\mathbf{N}}^{\prime }=A\mathbf{N}$ andThis identity needs finite second moments, not a Gaussian approximation. Raman spectral rebinning with conserved integrated counts describes the count-preserving map; retaining its full covariance matrix is necessary for uncertainties of sums and fitted spectral features.^[ref]

Consider independent input counts with means $100$ and $300$ and Poisson distribution variances equal to those means. Split them deterministically usingThe three output means are $50,200,150$. Their total has variance $1^T{C}^{\prime }1=400$, exactly the original total variance, and standard deviation $20$. Keeping only diagonal entries would give variance $200$ and standard deviation about $14.14$, understating the uncertainty of the conserved total.

The rank deficiency from deterministic spectral upsampling is visible without matrix inversion: the vector $(1,-1,1{)}^T$ has zero output contrast for every possible input. Three displayed channels therefore contain at most two independent degrees of freedom in this model. Fractional redistribution creates correlated numerical channels; it is different from physically assigning each detected photon randomly to one new channel.

The primary stimulated-Raman apparatus records spectra with array detectors and calibrates their spectral coordinates.^[ref] When those data are placed on a new spectral grid, the original detector counts remain the source of statistical information. Additional interpolation points do not supply independent photon measurements.

Rank deficiency from deterministic spectral upsampling occurs because a deterministic map cannot manufacture independent measurements. For $\mathbf{y}=A\mathbf{n}$ and input covariance matrix $C$,When the output has more channels than the input, its covariance matrix is necessarily singular. Covariance of count-conserving rebinned Raman channels gives the conserved-count example and its full propagated uncertainty.^[ref]

Using that parent's two-to-three channel map, the output always satisfies $y_1-y_2+y_3=0$. If the input readings are $20$ and $30$, the rebinned vector is $(10,25,15{)}^T$. Recovering the original readings as $n_1=2y_1$ and $n_2=2y_3$ shows that the third displayed number is not a third independent observation.

For an original fitting example, suppose both input means equal an unknown $\theta$ and their independent measurement-error variances are known to be $25$. Fitting in the original channels gives $\hat{\theta }=(20+30)/2=25$, with variance $25/2=12.5$. A correct generalized least squares fit after rebinning gives the same result and uncertainty. It must use the singular covariance on its supported range, for example through the Moore-Penrose inverse, together with the exact linear constraints.

The range restriction in singular-covariance spectral fitting matters if a proposed output model fails those constraints. A residual along $(1,-1,1{)}^T$ cannot arise from this rebinning model at all. Merely inserting a pseudoinverse into a quadratic objective would assign zero penalty to that impossible direction. Either enforce the range condition or supply a physically justified additional noise model; adding a small diagonal term solely to make the matrix invertible changes the statistical assumptions.

The motivating Raman apparatus measures detector-array spectra before any subsequent numerical grid change.^[ref] Working on the original grid often avoids this singular fitting problem while retaining the actual measurement information.

Range restriction in singular-covariance spectral fitting requires a residual to lie in the subspace where the measurement model allows random error. Let $\mathbf{y}=\mathbf{m}(\theta )+\mathbf{ϵ}$ with zero-mean error covariance $C$. Every vector $\mathbf{v}$ in the kernel of $C$ satisfiesso ${\mathbf{v}}^T\mathbf{ϵ}=0$ almost surely. Thus a permissible residual obeyswhere $C^{+}$ is the Moore-Penrose inverse. The matrix $C{C}^{+}$ is the orthogonal projector onto the range of the covariance.

Only after enforcing this condition should a generalized least squares objective use $(\mathbf{y}-\mathbf{m}{)}^T{C}^{+}(\mathbf{y}-\mathbf{m})$. The quadratic form alone assigns zero penalty to the forbidden directions and therefore cannot impose the condition. Rank deficiency from deterministic spectral upsampling gives its physical spectral-grid origin through $C=A{C}_{\mathrm{in}}{A}^T$. Covariance propagation preserves those exact dependencies.^[ref]

For an original idealized two-channel example, take $C=\mathrm{diag}(1,0)$, mean $\mathbf{m}(\theta )=(\theta ,2\theta {)}^T$ and observation $(3,4{)}^T$. The second channel has no random error in this stipulated model, so the range condition forces $\theta =2$. The remaining squared residual is one. Minimizing only the pseudoinverse quadratic would instead choose $\theta =3$ and objective zero, while predicting the impossible second-channel value six.

Real measurements need not contain a perfect channel: deterministic rebinning produces exact noise-free combinations of otherwise noisy channels. Additional physical noise in such a combination must be modelled if present. If covariance ranges change with the parameters, pseudoinverse fitting with a parameter-dependent covariance range requires enforcing the appropriate support for each candidate; a single fixed projection need not apply.

Raman detector-array spectra are recorded before numerical redistribution onto another spectral grid.^[ref] Fitting the original independent measurement coordinates often makes these support restrictions explicit and avoids treating interpolation points as new observations.

Pseudoinverse fitting with a parameter-dependent covariance range requires checking the allowed residual subspace separately for each fitted parameter. If the covariance is $C_{\theta }$ and the residual is ${\mathbf{r}}_{\theta }=\mathbf{y}-\mathbf{m}(\theta )$, the range restriction in singular-covariance spectral fitting becomesThe Moore-Penrose inverse quadratic ${\mathbf{r}}_{\theta }^T{C}_{\theta }^{+}{\mathbf{r}}_{\theta }$ alone does not enforce this condition. A projector computed at one trial parameter need not describe another trial's allowed noise.

For an original example, let ${\mathbf{u}}_{\theta }=(\cos \theta ,\sin \theta {)}^T$, zero mean and $C_{\theta }={\sigma }^2{\mathbf{u}}_{\theta }{\mathbf{u}}_{\theta }^T$. Noise lies on a line that rotates with $\theta$. With observation $\mathbf{y}=(2,0{)}^T$ and $\sigma =1$, the model at $\theta =0$ permits the observation and gives quadratic value four. At $\theta =\pi /2$, the pseudoinverse quadratic is zero, yet the observation is outside the allowed vertical line. Minimizing that quadratic without its range constraint would reward an impossible candidate.

The pinned SciPy documentation describes singular-covariance evaluation using pseudo-determinants, pseudoinverses and covariance rank; its implementation additionally applies a numerical support mask and returns zero density, or negative-infinite log density, off support.^[ref] Parameter-dependent supports also require care about which measure defines a density. Values defined on different subspaces are not automatically a common likelihood merely because each uses the same-looking formula.

A physically justified transverse noise resolves the idealized singularity. With perpendicular unit vector ${\mathbf{v}}_{\theta }$, use $C_{\theta ,ϵ}={\sigma }^2{\mathbf{u}}_{\theta }{\mathbf{u}}_{\theta }^T+{ϵ}^2{\mathbf{v}}_{\theta }{\mathbf{v}}_{\theta }^T$. For $ϵ=0.1$, the two quadratic values become four and four hundred. The determinant is the same at these orientations, so this full-rank Gaussian model correctly disfavors the transverse observation.

The transverse-noise limit of a rotating rank-one covariance model is an inference about a stated physical noise model, not permission to add arbitrary diagonal noise for numerical convenience. In spectral work, distinguish a parameter-dependent physical noise direction from a fixed deterministic rebinning matrix; the latter has a fixed output range when its input covariance is positive definite.

The transverse-noise limit of a rotating rank-one covariance model describes how an ordinary two-dimensional noise distribution concentrates onto a line as one physical noise scale tends to zero. For orthonormal vectors ${\mathbf{u}}_{\theta },{\mathbf{v}}_{\theta }$, setA zero-mean Gaussian observation has densityThe pseudoinverse fitting with a parameter-dependent covariance range article explains the range restriction at the exactly singular endpoint.

For a fixed point off the line, ${\mathbf{v}}_{\theta }^T\mathbf{y}\ne 0$, the exponential drives this density to zero as $ϵ\downarrow 0$. On the line the two-dimensional density grows like $1/ϵ$. There is no finite pointwise two-dimensional density limit: probability concentrates onto a one-dimensional Gaussian measure. A density on that line uses a different reference measure. Simply substituting zero for $ϵ$ in the full-rank density formula is invalid.

For an original angular example, let an observed vector have length $R=3$ along orientation ${\theta }_0$, with $\sigma =1$ and $ϵ=0.06$. At angular displacement $\delta =\theta -{\theta }_0$,At $\delta =0.02$ radians the density ratio is about $0.608$. The angular resolution set by transverse spectral noise has characteristic small-angle scale $ϵ/R\approx 0.02$ radians, or $1.15^{\circ }$, when the transverse noise is much smaller than the longitudinal noise.

SciPy's pinned implementation treats singular covariance using its rank and a separate numerical support check, rather than only discarding zero eigenvalues in a quadratic.^[ref] Numerical support tolerance and actual transverse measurement noise are different quantities. Choose the latter from the detector and processing model; a tiny arbitrary regularization can otherwise imply unjustified angular precision.

Angular resolution set by transverse spectral noise quantifies how sharply an observed spectral vector constrains the orientation of an anisotropic two-coordinate noise model. Use the covariance from transverse-noise limit of a rotating rank-one covariance model, with known longitudinal standard deviation $\sigma$ and transverse standard deviation $0<ϵ<\sigma$. For an observation of length $R>0$ aligned with orientation ${\theta }_0$, the Gaussian density givesThe determinant is independent of orientation, so the angular dependence comes entirely from the covariance-weighted residual.^[ref]

Define a local half-width by a chosen log-density drop $c>0$. For $2c\le K$, the lobe around ${\theta }_0$ has half-widthWhen this angle is small and $ϵ\ll \sigma$, it is approximately $\sqrt{2c}ϵ/R$. If $2c>K$, even an orthogonal orientation fails to reach that drop, so this local-width criterion does not exclude any orientation. At $R=0$ or equal noise scales, the observation has no angular preference in this model.

For an original example, choose $R=2$, $\sigma =0.8$ and $ϵ=0.04$ in consistent signal units. Then $K=2493.75$. A drop $c=1/2$ gives half-width about $0.0200$ radians, or $1.15^{\circ }$. Reducing only the transverse standard deviation to $0.02$ approximately halves the width. Increasing a numerical regularization parameter is not an experimental improvement in this noise scale.

The global orientation ambiguity in a rank-one covariance fit remains: orientations separated by $\pi$ give the same covariance. A narrow local lobe does not distinguish those two orientations. Nor is a fixed log-density drop automatically a confidence interval with a stated coverage probability. Multiple observations, unknown noise scales and experimental nuisance parameters require their corresponding inference model.

Repeated reference spectra under stable conditions can estimate the longitudinal and transverse noise components. Their covariance must include shared calibration effects, as required by the measurement-propagation rule.^[ref] The numerical example is a model calculation in spectral coordinates, rather than a specification of a particular optical instrument.

Global orientation ambiguity in a rank-one covariance fit means that covariance identifies an axis without identifying its positive direction. For ${\mathbf{u}}_{\theta }=(\cos \theta ,\sin \theta {)}^T$ and $\lambda >0$,because ${\mathbf{u}}_{\theta +\pi }=-{\mathbf{u}}_{\theta }$. Adding an isotropic transverse-noise term does not remove this equality. The angular resolution set by transverse spectral noise can make each local orientation lobe narrow while leaving the two directed orientations indistinguishable.

For an original example, let $\lambda =4$ and $\theta =30^{\circ }$. Both $30^{\circ }$ and $210^{\circ }$ giveThe eigenvalues are four and zero, and the principal eigenvector can be written as either $(\sqrt{3}/2,1/2{)}^T$ or its negative. More samples can improve estimation of this matrix without choosing between those signs.

In a zero-mean Gaussian covariance model, equal covariance gives equal probability laws, including the supported line in the singular case.^[ref] No fitting procedure using only that law can distinguish the two parameter labels. A fitting range $0\le \theta <\pi$ removes duplicate labels by convention; it does not measure the sign of a physical direction.

For example, local high-likelihood neighborhoods around $30^{\circ }$ and $210^{\circ }$ may each span only a few degrees. Reporting only the first as uncertainty in a directed angle discards an equally supported region unless additional information rules it out. A prior or coordinate convention that selects one sign should be distinguished from an experimental observation.

Sign-sensitive measurements that resolve a covariance-axis ambiguity can supply information outside the covariance-only model, such as an independently calibrated signed mean or phase reference. If the scientific quantity is genuinely an unoriented axis, an angle modulo $\pi$ is already the appropriate parameter. The interpretation of the parameter determines whether the ambiguity needs resolving.

Sign-sensitive measurements that resolve a covariance-axis ambiguity add an observable whose distribution changes when the fitted axis direction is reversed. In the global orientation ambiguity in a rank-one covariance fit, $\mathbf{u}$ and $-\mathbf{u}$ give the same covariance. An independently calibrated signed projection can distinguish them.

Consider two candidates with an additional observation $z=\pm m+\eta$, where $m=a{\mathbf{w}}^T\mathbf{u}\ne 0$ is known, $a$ has known positive sign and $\eta \sim N(0,s^2)$. Assume this calibration reading is independent of the covariance data conditional on the candidate. The Gaussian densities giveThe normal density and cumulative distribution function used here are documented in the pinned numerical reference.^[ref] The likelihood-ratio expression follows by expanding the two squared residuals.

For an original example, let $m=0.20\mathrm{V}$, $s=0.050\mathrm{V}$ and observe $z=0.10\mathrm{V}$. The log likelihood ratio is 16, strongly favoring the positive candidate under this stated model. With equal prior probabilities and equal decision costs, the sign-threshold decision has repeated-measurement error probability $\Phi (-|m|/s)=\Phi (-4)\approx 3.17\times 10^{-5}$. This decision-error probability is different from assigning a posterior error probability to that one observed value.

A practical calibration applies a known signed perturbation and records the signed detector-response component. Reversing the applied perturbation checks readout polarity. A calibrated phase-sensitive readout can provide such a projection; measuring only noise covariance does not supply its sign.

The failure of sign recovery from magnitude-only calibration data is important. For symmetric Gaussian noise, $|z|$ has the same distribution under means $+m$ and $-m$. If $\mathbf{w}$ is orthogonal to the axis, then $m=0$ and even the signed reading adds no separation. If the amplitude sign itself is unknown, reversing both amplitude and axis preserves the mean. The additional measurement resolves the ambiguity only when its reference sign and response model are independently established.

Failure of sign recovery from magnitude-only calibration data occurs when opposite signed calibration models induce exactly the same distribution of every recorded magnitude. Under this symmetry, more repetitions can estimate the magnitude distribution accurately without deciding the missing sign. This is the limitation relevant to sign-sensitive measurements that resolve a covariance-axis ambiguity.

Let $Z=sm+ϵ$ with sign $s\in \{-1,+1\}$, known $m>0$ and symmetric noise $ϵ$ having density $f(x)=f(-x)$. If the recorded value is $Y=|Z|$, then for $y>0$Symmetry gives $g(y\mid +1)=f(y-m)+f(y+m)=g(y\mid -1)$. Independent repetitions multiply identical densities, so their likelihood ratio remains one. With equal sign priors and equal decision costs, no decision rule based solely on these magnitudes can have average sign-error probability below1/2.

For normal distribution noise with standard deviation $\sigma$, the magnitude has a folded normal distribution:where $\varphi$ is the standard normal density. The formula depends on $|m|$, not its sign. The pinned numerical reference provides the equivalent standardized folded-normal density and its scale transformation.^[ref]

For the original calibration scale $m=0.20\mathrm{V}$ and $\sigma =0.050\mathrm{V}$, recording a signed value can distinguish the two models strongly, but recording only its magnitude cannot do so under the symmetric model. This is an information loss in the measurement map, rather than a shortage of averaging trials.

The symmetry assumption is essential. As an original counterexample, let noise equal−1 with probability2/3 and+2 with probability1/3, giving zero mean but an asymmetric distribution. With $m=1$, the positive-sign magnitudes are0 or3, while the negative-sign magnitudes are2 or1. Their disjoint supports permit sign recovery if this asymmetric noise model is independently established. The calibrated asymmetry itself supplies sign-reference information absent from the symmetric case.

Similarly, a genuinely signed projection or a known phase reference can break the original ambiguity. An unverified readout polarity or unknown amplitude sign can preserve it. Specify which signed information the apparatus actually records before interpreting a magnitude fit as an oriented calibration.