With known covariance parameters, best linear unbiased prediction minimizes prediction-error variance over linear predictors that are unbiased over both observation errors and random effects, for every fixed-effect value. In a Gaussian linear mixed model, the random-effect predictor is , where generalized least squares estimates the fixed effects. Plugging in covariance estimates gives an empirical predictor; its uncertainty must also account for estimating those covariance parameters. It differs from a best linear unbiased estimator of an unknown fixed coefficient.
Alvarez describes a progression from measured coordinates to momenta and angles, then conservation-constrained fits testing alternative track identities. Computer programs made the repeated tests practical; an insufficiently constrained event could retain several hypotheses.[ref]
A mathematical formulation explains what such a fit can establish. Let collect measured momenta and angles, with positive-definite covariance matrix . For a chosen reaction hypothesis, adjust them to by minimizing a generalized least squares objective,
The constraints express four-momentum conservation and any independently known daughter masses. For each proposed identity, calculate energy from the energy–momentum relation, . A missing neutral particle adds unknowns; a measured decay vertex can add geometric constraints. Compatibility depends on the measurement errors and the independent constraints, not just the smallest raw among unrelated fits.
A simple independent check uses a single momentum conservation constraint. Suppose two measured one-dimensional momenta are and , each with independent standard deviation , and a stipulated parent at rest requires their sum to vanish. Minimizing the two equal-weight squared changes gives fitted values and , with . A fit repairs a mismatch according to a hypothesis; it does not prove that hypothesis. This one-component illustration is not a complete reaction fit.
A correlated-channel update of pressure-map information measures what a new pressure channel adds after accounting for noise that it shares with the old channels. The rank-one update of pressure-map information for independent noise is a special case. An extra channel may be a precise-looking duplicate of an existing noisy signal and contribute almost no new information.
Write the old and new observations as and . Suppose their known joint error covariance matrix is positive definite,
Subtract the part predicted from old noise by defining
Direct multiplication gives and , a Schur complement. Thus the invertible residual transformation separates the weighted residual objective into its old term and one scalar innovation term:
For jointly normal random variables, uncorrelatedness here also gives independence, and the matrix is the parameter Fisher information. For more general zero-mean errors with this fixed covariance, the displayed algebra still describes the generalized least squares information matrix. This is a partial whitening transformation; the primary covariance interface documents whitening and its identity-covariance check.[ref]
An original scalar example observes the same pressure twice, with unit error variances and error correlation . Then , , so
At , the true estimator variance is , whereas an independent-noise calculation would claim . At , the variance is : opposite noise contributions can cancel. Correlation can reduce or increase the benefit relative to the independent case; it does not make the correctly computed added information negative.
The effective sensitivity is , not merely the original . If and , the new residual carries no additional pressure information. A zero-innovation-variance pressure channel requires separate treatment: division by is invalid, and the limiting relation may be a redundant duplicate or an exact constraint, depending on .
A proposed array characterization measures simultaneous repeated readings under fixed known loads, estimates the complete cross-channel covariance, and tests reconstruction on withheld repetitions. A common reference offset can survive indefinitely despite more channels. The derivation assumes known, unchanged sensitivities and covariance; estimating them from few samples adds uncertainty not included by this update.
Gauss-Markov theorem Created 2026-09-24 Updated 2026-11-05
Let with a fixed full column rank design matrix, and . Then ordinary least squares is a best linear unbiased estimator: every response-linear unbiased estimator has at least as much variance in every coefficient direction. A normal distribution is unnecessary.[ref]
For a proof put . A competing estimator is unbiased for every exactly when . Write ; then , so the cross terms vanish and
This positive semidefinite matrix inequality proves the simultaneous directional comparison. The scope is linear unbiased estimators, not all conceivable estimators: a biased ridge regression estimate may improve mean squared error for some signals. With correlated or unequal-variance errors, the corresponding known-covariance result uses generalized least squares.
Inverse-variance weighting combines estimates of the same quantity by giving a more precise estimate a larger weight. Suppose are independent, unbiased estimators of , with known variances . For normalized weights,
The inverse is taken after squaring the standard uncertainty: twice the standard uncertainty gives one quarter as much weight. The corresponding weighted least squares criterion is ; differentiating it with respect to gives the same weighted arithmetic mean.[ref]
The optimality statement is precise. Any linear unbiased estimator requires . By the Cauchy-Schwarz inequality,
Equality holds for the displayed weights. Thus they minimize variance among linear unbiased combinations under these assumptions; a normal distribution is not needed for that proof.
For an original example, combine observed values 10 and 14 with standard uncertainties 1 and 2 in the same units. The normalized weights are and , giving estimate , variance and standard uncertainty approximately . Equal weights give 12 and standard uncertainty .
Correlated errors change the problem. A contribution common to every reading survives every weighting whose coefficients sum to one; precision limits of weighting correlated repeats derives the resulting floor. More generally, use the full covariance matrix and generalized least squares rather than treating correlated readings as independent.[ref] Uncertainty in estimated inverse-variance weights is another limitation: variances estimated from the same data are not fixed known quantities, and an accidentally tiny estimated variance does not justify arbitrary confidence.
Minimum-noise weights under baseline moment constraints choose a linear pulse estimator with the smallest variance among those that preserve a specified signal and remove specified backgrounds. Write the reading vector as , where is a known signal template, the columns of contain baseline shapes, and the error covariance matrix is positive definite. For , require
Columns in implement the moment conditions for baseline-drift cancellation. The noise cost of polynomial baseline cancellation depends on which feasible weights are chosen.
If the constraint columns are independent, Lagrange multipliers give
For any feasible perturbation , and , so the variance increases by . This proves the minimum and connects the estimator to generalized least squares. The covariance propagation is the full quadratic form, not a sum that discards correlations.[ref]
For an original one-bin pulse example, use five equally spaced times , a signal present only at the centre, and independent errors of variance . To remove just constant and linear backgrounds, the optimal dimensionless weights are . Their variance is , lower than for the centre and its two nearest references alone. Multiplying the amplitude estimate by a bin duration gives energy variance . Extra uncontaminated references can reduce noise when the cancelled background family stays the same.
The inverse requires genuinely independent constraints. Constraint redundancy in baseline-noise minimization calls for removing redundant columns and checking feasibility; a signal template inside the nuisance span cannot be preserved while that span is rejected. NIST's before/after pulse baselines provide the physical measurement context.[ref] The five-point design is an original idealization and assumes no pulse signal in its reference samples.
Multivariate calibration 2026-11-05
Multivariate calibration relates several measured signals jointly to one or more chemical quantities, using reference samples whose relevant properties are known. Multiple wavelengths, detector channels or other signal coordinates can help distinguish overlapping component responses. The IUPAC multicomponent-calibration report treats both forward response models and inverse prediction, together with uncertainty and model validation.[ref]
In a forward linear measurement model, write
where contains analyte concentrations, is a background response, and contains channel sensitivities determined from calibration mixtures. If has full column matrix rank, an unweighted least-squares estimator is
The number of channels alone does not guarantee identification: their response vectors must provide independent information.
For an original example with fixed chosen signal/concentration units, set
The estimated concentrations are . Predicted signals are , so the calibration residuals are . They satisfy : least squares balances the three channels rather than satisfying the first two exactly. This example treats the calibrated sensitivities as known; their uncertainty would add another contribution.
If the channel errors instead have known covariance matrix , generalized least squares replaces the estimate by
Correlated channels do not provide the same information as independent repetitions. Nearly dependent sensitivity columns also amplify noise, connecting wavelength selection with the condition number of the inverse problem.
An inverse model can instead fit concentrations directly from measured signal vectors. Methods such as principal component regression reduce the signal representation, but the retained variation must still predict the intended chemical quantity. Cross-validation and separately prepared mixtures test prediction; a high calibration fit alone cannot establish transfer to another sample matrix. Preparing standards in which component concentrations vary independently helps expose confounding between analytes, rather than teaching the model only one fixed mixture ratio.
Assume the known covariance matrix is positive definite. The Gaussian likelihood, up to constants, is
Its gradient is . The negative Hessian matrix is positive definite because has full column rank. Thus the unique maximum-likelihood estimator is the generalized least squares estimator
It is unbiased, and multiplication of its linear coefficient matrix by gives
In fact . Equivalently, a whitening transformation premultiplies and by , after which ordinary least squares applies. There is no extra factor when is the complete known covariance; such a factor appears if only a shape matrix is known and .
The restricted maximum likelihood procedure estimates covariance parameters from error contrasts that remove the unknown fixed effects. Let have rank and choose an full-rank matrix with . Then has a likelihood involving the variance parameters but not . Maximizing that likelihood is restricted maximum likelihood. An equivalent restricted likelihood from orthogonal error contrasts, up to terms independent of , is
where
After estimating , use generalized least squares for and the conditional modes from part (b). In an independent-error normal linear model, this covariance procedure reduces to estimating by instead of .
The two models being compared have different fixed-effect spaces, so their error contrasts, dimensions, and restricted likelihoods differ. Their REML values are not likelihoods of the same transformed data and must not be used directly for this fixed-effect likelihood-ratio test. Refit both with ordinary maximum likelihood, integrating out the random effects but retaining the original response vector, and maximize each model's marginal likelihood.
The null hypothesis is , leaving the random slope variance free; the alternative permits a nonzero population stirring effect. The supplied test has , one extra fixed coefficient, and approximate p-value . Retain the fixed stirring-rate effect at 5%. A mean-zero random slope cannot substitute for an estimated nonzero population slope. The asymptotic calibration assumes regular nuisance covariance parameters and should be interpreted cautiously with only three furnaces; simulation can provide a better finite-sample calibration.
Let have columns , and let have one column per incubator, with entry in the column for observation 's incubator and zeros elsewhere. Integrating the Gaussian random effects gives
Consequently its marginal log-likelihood is
The likelihood-ratio test of against uses
For each covariance, the optimizing coefficients are the generalized least squares estimator . Both fits should use the same marginal likelihood convention, for example fitting the alternative with update(fly.model, REML=FALSE) and the null as a Gaussian linear model. The displayed alternative REML criterion alone cannot supply .
The variance component is constrained to be nonnegative, and zero lies on the boundary. Thus the interior-parameter hypothesis of Wilks theorem fails; the usual approximation is inappropriate. A limit sometimes applies to a single variance component under additional asymptotic conditions, but four incubators do not give a persuasive large-number-of-groups approximation.
Use a parametric bootstrap under the null: estimate there; simulate independent Gaussian errors with that fitted variance at exactly the observed times and grouping labels; refit both models by maximum likelihood for each simulated sample; and calculate . The upper-tail proportion, conventionally , estimates the p-value. Boundary fits with estimated must remain in the simulation distribution. A restricted-likelihood ratio and a correspondingly calibrated bootstrap are another option when the fixed-effect design is identical, but one must not mix ordinary and restricted likelihoods in a single statistic.
Let have rows and let be the spatial covariance matrix with . Conditional on chosen covariance parameters, the best linear unbiased estimator of the drift coefficients is the generalized least squares estimator
The printed procedure first estimates the linear drift from depth ~ y, constructs a binned residual empirical semivariogram, fits a Gaussian semivariogram to it, and then uses the fitted covariance in generalized least squares trend estimation and universal kriging. The missing fit could be supplied as
gauss.model2 <- fit.variogram(smvg2, vgm(0.5, "Gau", 1, 0))
where those arguments are initial values, not the reported final estimates. Such semivariogram fitting typically minimizes a weighted sum of squared discrepancies between binned empirical values and model values; it is not automatically a joint Gaussian likelihood maximization. One can iterate: form residuals using the current drift, refit the covariance model, recompute the generalized least squares drift, and repeat until changes are small. The displayed commands themselves show a single sequence, not evidence that such iteration occurred.
A likelihood-based alternative estimates the two blocks coherently from
Substitute to obtain a profile log-likelihood, maximize over and with a positive-definite matrix , and recover the drift coefficients from the optimizing covariance. Restricted maximum likelihood adds the design determinant term and uses in place of in the normalizing term, up to a fixed-design constant. It can reduce bias from estimating the drift before the covariance. A zero fitted nugget is a legitimate boundary estimate, not proof that measurement error is absent. Estimate drift using covariance-weighted regression, and spatial dependence from drift-adjusted variation, with the two stages or likelihood explicitly distinguished.
The log likelihood, up to constants, is
Differentiation gives the generalized least squares estimator
and maximizing over the scale gives
The divisor is appropriate for maximum likelihood rather than unbiased estimation.
Range restriction in singular-covariance spectral fitting requires a residual to lie in the subspace where the measurement model allows random error. Let with zero-mean error covariance . Every vector in the kernel of satisfies
so almost surely. Thus a permissible residual obeys
where is the Moore-Penrose inverse. The matrix is the orthogonal projector onto the range of the covariance.
Only after enforcing this condition should a generalized least squares objective use . The quadratic form alone assigns zero penalty to the forbidden directions and therefore cannot impose the condition. Rank deficiency from deterministic spectral upsampling gives its physical spectral-grid origin through . Covariance propagation preserves those exact dependencies.[ref]
For an original idealized two-channel example, take , mean and observation . The second channel has no random error in this stipulated model, so the range condition forces . The remaining squared residual is one. Minimizing only the pseudoinverse quadratic would instead choose and objective zero, while predicting the impossible second-channel value six.
Real measurements need not contain a perfect channel: deterministic rebinning produces exact noise-free combinations of otherwise noisy channels. Additional physical noise in such a combination must be modelled if present. If covariance ranges change with the parameters, pseudoinverse fitting with a parameter-dependent covariance range requires enforcing the appropriate support for each candidate; a single fixed projection need not apply.
Raman detector-array spectra are recorded before numerical redistribution onto another spectral grid.[ref] Fitting the original independent measurement coordinates often makes these support restrictions explicit and avoids treating interpolation points as new observations.
Rank deficiency from deterministic spectral upsampling occurs because a deterministic map cannot manufacture independent measurements. For and input covariance matrix ,
When the output has more channels than the input, its covariance matrix is necessarily singular. Covariance of count-conserving rebinned Raman channels gives the conserved-count example and its full propagated uncertainty.[ref]
Using that parent's two-to-three channel map, the output always satisfies . If the input readings are and , the rebinned vector is . Recovering the original readings as and shows that the third displayed number is not a third independent observation.
For an original fitting example, suppose both input means equal an unknown and their independent measurement-error variances are known to be . Fitting in the original channels gives , with variance . A correct generalized least squares fit after rebinning gives the same result and uncertainty. It must use the singular covariance on its supported range, for example through the Moore-Penrose inverse, together with the exact linear constraints.
The range restriction in singular-covariance spectral fitting matters if a proposed output model fails those constraints. A residual along cannot arise from this rebinning model at all. Merely inserting a pseudoinverse into a quadratic objective would assign zero penalty to that impossible direction. Either enforce the range condition or supply a physically justified additional noise model; adding a small diagonal term solely to make the matrix invertible changes the statistical assumptions.
The motivating Raman apparatus measures detector-array spectra before any subsequent numerical grid change.[ref] Working on the original grid often avoids this singular fitting problem while retaining the actual measurement information.
RNA log-odds covariance 2026-11-05
RNA log-odds covariance describes correlated errors between transformed population estimates at different temperatures in an RNA melting log-odds plot. Shared RNA melting baselines can generate it even when individual detector readings are independent.
Let be unfolded fractions and . First-order propagation gives
In matrix form, with diagonal . This is the multivariate form of RNA melting log-odds uncertainty; the derivatives are evaluated at the estimated fractions.[ref]
A concrete shared-input model uses . If all folded-state predictions contain the same uncertain offset , then
This component has one shared error direction. It is not equivalent to adding independent noise with variance to every observation. Other baseline parameters contribute through RNA baseline prediction covariance.
For an original simplified example, nine observations each have independent log-odds standard uncertainty0.03 and the same shared-offset contribution 0.01. The standard uncertainty of their mean is
Incorrectly treating the shared term as independent would give . Repetition reduces the independent component while leaving this common offset contribution intact.
A model fit can use the joint covariance matrix in generalized least squares, with an objective of the form when that inverse exists. Alternatively, fit the underlying absorbance and baseline parameters jointly. Shared-error components alone may be singular, and near population endpoints the logarithmic linearization may fail. A precise straight-line fit does not independently validate the two-state RNA melting model or its baseline assumptions.
A tie-line phase-fraction fit estimates how much of each liquid phase is needed to reproduce an overall composition from specified liquid-liquid tie line endpoints. If are endpoint mole fraction vectors and is the mole-based phase fraction of phase , the mass balance predicts
The endpoints must be supplied by equilibrium measurements or a suitable thermodynamic model; fitting does not establish that an arbitrary pair of compositions is an equilibrium tie line.[ref]
For a ternary liquid-liquid tie line, use two independent composition coordinates, such as the first two mole fractions, since the third is fixed by their sum. Let be these two measured coordinates, the corresponding phase- coordinates, the difference between phase- and phase- coordinates, and a positive-definite covariance matrix for . With exact endpoints, generalized least squares minimizes
Differentiation yields the unconstrained estimate
The second expression imposes the physical bounds. It follows because is a convex quadratic when the endpoints differ in the measured coordinates. The ternary tie-line consistency residual then records the remaining failure of the mass balance.
For an original hypothetical example, take endpoint compositions and , with overall measurement . Suppose the first two measured mole fractions have independent standard measurement uncertainties and , so . Then
The predicted composition is , giving residual . The separate first- and second-component balances would suggest and ; the fit combines them according to their stated measurement uncertainties, rather than averaging those two numbers without regard to precision.
With Gaussian composition errors, fixed endpoints and an interior unconstrained fit, the tie-line phase-fraction uncertainty follows directly from the linear estimator:
for this example. Endpoint uncertainty, a fit on a boundary, or a nearly vanishing endpoint difference needs additional treatment. In particular, all three normalized mole fractions must not be treated as independent observations with an invertible three-coordinate covariance matrix: their exact closure creates dependence. A small fitted residual is evidence of consistency with the specified mass balance and error model, not a substitute for testing chemical potential equality or phase stability.
Tie-line phase-fraction sensitivity measures how an inferred phase fraction changes when the overall composition or the measured liquid-liquid tie line endpoints change. A sensitivity coefficient is a local partial derivative of the inference formula, rather than an uncertainty by itself. It becomes an uncertainty contribution only after combination with the input covariance matrix.[ref]
For one composition coordinate of a binary mixture, write the lever rule as
where are endpoint mole fractions, is the overall mole fraction and is the mole-based amount of phase . Differentiating the mass balance gives
Thus a measurement error in the overall composition and an equal measurement error in the phase- endpoint have different effects. The conditioning of tie-line phase-fraction inference deteriorates as the endpoint separation becomes small: the same composition measurement error then produces a larger change in the inferred phase fraction.
For an original example, take , , , giving . Increasing only by gives a first-order change . Direct substitution gives , a change of . For the closer endpoints , and balanced composition , the local sensitivity to the same endpoint error is ten times larger. This is a property of the inference geometry; it does not establish whether either hypothetical pair is in liquid-liquid equilibrium.
The multidimensional tie-line phase-fraction fit has an additional detail away from an exactly balanced composition. On independent mole fraction coordinates, hold a symmetric positive-definite matrix fixed and define
with distinct endpoints. Define row vectors and . Differentiating the generalized least squares normal equation gives
The residual correction in tie-line sensitivity vanishes when . Otherwise, moving an endpoint also rotates the fitted line relative to the nonzero ternary tie-line consistency residual. Omitting that contribution is an additional approximation. If itself changes with the measured inputs, its derivative must also enter the calculation.
These formulas describe the unconstrained fit locally. Imposing introduces boundary effects, and coincident endpoints make this inference undefined. A practical sensitivity check recomputes the fit after small positive and negative perturbations of one measured coordinate, keeping composition closure and the stated weight matrix. Agreement as the perturbation shrinks tests the derivative calculation; it does not validate the underlying equilibrium model.
Figure 1.
Closer tie-line endpoints amplify composition errors; endpoint correlation changes the inferred phase-fraction uncertainty
. Original binary examples with phase fraction0.40. Left: exact endpoint perturbations at separations0.60 and0.06. Right: first-order standard uncertainty with uncertain endpoints and exact overall composition; common calibration of all three measurements is a different model. Details.
Universal kriging Created 2026-10-05 Updated 2026-10-18
Universal kriging allows an unknown regression drift:
The design matrix has full column rank, its entries and are known, and the regression coefficients are unknown. For example, rows and target row allow a linear spatial trend. Requiring to have zero bias for every gives . With known positive-definite matrix observation covariance matrix , the minimizing weights solve
where is a vector of Lagrange multipliers. Equivalently, the generalized least squares estimate and prediction are
The first term is the estimated trend alone; the second predicts the correlated regression residuals. With , the error variance is
The last, nonnegative term accounts for estimating the specified drift coefficients. Merely inserting a fitted drift into simple kriging and keeping its known-mean error variance would omit this term. For a Gaussian random field, the same expressions arise by integrating over drift coefficients in a diffuse prior distribution limit; this is an additional inferential interpretation, not an assumption required by the linear optimization.[ref]