Best linear unbiased prediction 2026-10-06
With known covariance parameters, best linear unbiased prediction minimizes prediction-error variance over linear predictors that are unbiased over both observation errors and random effects, for every fixed-effect value. In a Gaussian linear mixed model, the random-effect predictor is , where generalized least squares estimates the fixed effects. Plugging in covariance estimates gives an empirical predictor; its uncertainty must also account for estimating those covariance parameters. It differs from a best linear unbiased estimator of an unknown fixed coefficient.
Bubble-chamber kinematic fitting 2026-10-18
Alvarez describes a progression from measured coordinates to momenta and angles, then conservation-constrained fits testing alternative track identities. Computer programs made the repeated tests practical; an insufficiently constrained event could retain several hypotheses.[ref]
Recent Developments in Particle Physics by Luis Walter Alvarez. UCRL-18696, printed pp. 30–33: HYDRO, PANG, GUTS, KICK and SUMX
A mathematical formulation explains what such a fit can establish. Let collect measured momenta and angles, with positive-definite covariance matrix . For a chosen reaction hypothesis, adjust them to by minimizing a generalized least squares objective,The constraints express four-momentum conservation and any independently known daughter masses. For each proposed identity, calculate energy from the energy–momentum relation, . A missing neutral particle adds unknowns; a measured decay vertex can add geometric constraints. Compatibility depends on the measurement errors and the independent constraints, not just the smallest raw among unrelated fits.
A simple independent check uses a single momentum conservation constraint. Suppose two measured one-dimensional momenta are and , each with independent standard deviation , and a stipulated parent at rest requires their sum to vanish. Minimizing the two equal-weight squared changes gives fitted values and , with . A fit repairs a mismatch according to a hypothesis; it does not prove that hypothesis. This one-component illustration is not a complete reaction fit.
Gauss-Markov theorem Created 2026-09-24 Updated 2026-11-05
Let with a fixed full column rank design matrix, and . Then ordinary least squares is a best linear unbiased estimator: every response-linear unbiased estimator has at least as much variance in every coefficient direction. A normal distribution is unnecessary.[ref]
The Truth about Linear Regression by Cosma Rohilla Shalizi. Chapter 21, section 21.3, printed pp. 363–365
For a proof put . A competing estimator is unbiased for every exactly when . Write ; then , so the cross terms vanish andThis positive semidefinite matrix inequality proves the simultaneous directional comparison. The scope is linear unbiased estimators, not all conceivable estimators: a biased ridge regression estimate may improve mean squared error for some signals. With correlated or unequal-variance errors, the corresponding known-covariance result uses generalized least squares.
Inverse-variance weighting 2026-11-05
Inverse-variance weighting combines estimates of the same quantity by giving a more precise estimate a larger weight. Suppose are independent, unbiased estimators of , with known variances . For normalized weights,The inverse is taken after squaring the standard uncertainty: twice the standard uncertainty gives one quarter as much weight. The corresponding weighted least squares criterion is ; differentiating it with respect to gives the same weighted arithmetic mean.[ref]
The optimality statement is precise. Any linear unbiased estimator requires . By the Cauchy-Schwarz inequality,Equality holds for the displayed weights. Thus they minimize variance among linear unbiased combinations under these assumptions; a normal distribution is not needed for that proof.
For an original example, combine observed values 10 and 14 with standard uncertainties 1 and 2 in the same units. The normalized weights are and , giving estimate , variance and standard uncertainty approximately . Equal weights give 12 and standard uncertainty .
Correlated errors change the problem. A contribution common to every reading survives every weighting whose coefficients sum to one; precision limits of weighting correlated repeats derives the resulting floor. More generally, use the full covariance matrix and generalized least squares rather than treating correlated readings as independent.[ref]
Guide to the expression of uncertainty in measurement by JCGM. §5.2.2, equation13, and §5.2.4, common inputs
Uncertainty in estimated inverse-variance weights is another limitation: variances estimated from the same data are not fixed known quantities, and an accidentally tiny estimated variance does not justify arbitrary confidence.Minimum-noise weights under baseline moment constraints choose a linear pulse estimator with the smallest variance among those that preserve a specified signal and remove specified backgrounds. Write the reading vector as , where is a known signal template, the columns of contain baseline shapes, and the error covariance matrix is positive definite. For , requireColumns in implement the moment conditions for baseline-drift cancellation. The noise cost of polynomial baseline cancellation depends on which feasible weights are chosen.
If the constraint columns are independent, Lagrange multipliers giveFor any feasible perturbation , and , so the variance increases by . This proves the minimum and connects the estimator to generalized least squares. The covariance propagation is the full quadratic form, not a sum that discards correlations.[ref]
For an original one-bin pulse example, use five equally spaced times , a signal present only at the centre, and independent errors of variance . To remove just constant and linear backgrounds, the optimal dimensionless weights are . Their variance is , lower than for the centre and its two nearest references alone. Multiplying the amplitude estimate by a bin duration gives energy variance . Extra uncontaminated references can reduce noise when the cancelled background family stays the same.
The inverse requires genuinely independent constraints. Constraint redundancy in baseline-noise minimization calls for removing redundant columns and checking feasibility; a signal template inside the nuisance span cannot be preserved while that span is rejected. NIST's before/after pulse baselines provide the physical measurement context.[ref]
Calibration Service for Low-level Pulsed-Laser Radiometers at 1.06 µm: Pulse Energy and Peak Power by Rodney W. Leonhardt. §3.5, printed p. 8; §7.2, Leakage Effect
The five-point design is an original idealization and assumes no pulse signal in its reference samples. Multivariate calibration 2026-11-05
Multivariate calibration relates several measured signals jointly to one or more chemical quantities, using reference samples whose relevant properties are known. Multiple wavelengths, detector channels or other signal coordinates can help distinguish overlapping component responses. The IUPAC multicomponent-calibration report treats both forward response models and inverse prediction, together with uncertainty and model validation.[ref]
Guidelines for calibration in analytical chemistry. Part 2: Multicomponent calibration by Klaus Danzer and others. Sections 2–5; official project description, regression and validation scope
In a forward linear measurement model, writewhere contains analyte concentrations, is a background response, and contains channel sensitivities determined from calibration mixtures. If has full column matrix rank, an unweighted least-squares estimator isThe number of channels alone does not guarantee identification: their response vectors must provide independent information.
For an original example with fixed chosen signal/concentration units, setThe estimated concentrations are . Predicted signals are , so the calibration residuals are . They satisfy : least squares balances the three channels rather than satisfying the first two exactly. This example treats the calibrated sensitivities as known; their uncertainty would add another contribution.
If the channel errors instead have known covariance matrix , generalized least squares replaces the estimate byCorrelated channels do not provide the same information as independent repetitions. Nearly dependent sensitivity columns also amplify noise, connecting wavelength selection with the condition number of the inverse problem.
An inverse model can instead fit concentrations directly from measured signal vectors. Methods such as principal component regression reduce the signal representation, but the retained variation must still predict the intended chemical quantity. Cross-validation and separately prepared mixtures test prediction; a high calibration fit alone cannot establish transfer to another sample matrix. Preparing standards in which component concentrations vary independently helps expose confounding between analytes, rather than teaching the model only one fixed mixture ratio.
Past exam of the mathematics course of the University of Cambridge 2016 iii Paper 206 2 f Solution 2026-10-06
Assume the known covariance matrix is positive definite. The Gaussian likelihood, up to constants, isIts gradient is . The negative Hessian matrix is positive definite because has full column rank. Thus the unique maximum-likelihood estimator is the generalized least squares estimatorIt is unbiased, and multiplication of its linear coefficient matrix by givesIn fact . Equivalently, a whitening transformation premultiplies and by , after which ordinary least squares applies. There is no extra factor when is the complete known covariance; such a factor appears if only a shape matrix is known and .
Past exam of the mathematics course of the University of Cambridge 2016 iii Paper 206 6 c Solution Created 2026-10-03 Updated 2026-10-06
The restricted maximum likelihood procedure estimates covariance parameters from error contrasts that remove the unknown fixed effects. Let have rank and choose an full-rank matrix with . Then has a likelihood involving the variance parameters but not . Maximizing that likelihood is restricted maximum likelihood. An equivalent restricted likelihood from orthogonal error contrasts, up to terms independent of , iswhereAfter estimating , use generalized least squares for and the conditional modes from part (b). In an independent-error normal linear model, this covariance procedure reduces to estimating by instead of .
The two models being compared have different fixed-effect spaces, so their error contrasts, dimensions, and restricted likelihoods differ. Their REML values are not likelihoods of the same transformed data and must not be used directly for this fixed-effect likelihood-ratio test. Refit both with ordinary maximum likelihood, integrating out the random effects but retaining the original response vector, and maximize each model's marginal likelihood.
The null hypothesis is , leaving the random slope variance free; the alternative permits a nonzero population stirring effect. The supplied test has , one extra fixed coefficient, and approximate p-value . Retain the fixed stirring-rate effect at 5%. A mean-zero random slope cannot substitute for an estimated nonzero population slope. The asymptotic calibration assumes regular nuisance covariance parameters and should be interpreted cautiously with only three furnaces; simulation can provide a better finite-sample calibration.
Past exam of the mathematics course of the University of Cambridge 2017 iii Paper 206 4 b Solution Created 2026-10-03 Updated 2026-10-05
Let have columns , and let have one column per incubator, with entry in the column for observation 's incubator and zeros elsewhere. Integrating the Gaussian random effects givesConsequently its marginal log-likelihood isThe likelihood-ratio test of against usesFor each covariance, the optimizing coefficients are the generalized least squares estimator . Both fits should use the same marginal likelihood convention, for example fitting the alternative with
update(fly.model, REML=FALSE) and the null as a Gaussian linear model. The displayed alternative REML criterion alone cannot supply .The variance component is constrained to be nonnegative, and zero lies on the boundary. Thus the interior-parameter hypothesis of Wilks theorem fails; the usual approximation is inappropriate. A limit sometimes applies to a single variance component under additional asymptotic conditions, but four incubators do not give a persuasive large-number-of-groups approximation.
Use a parametric bootstrap under the null: estimate there; simulate independent Gaussian errors with that fitted variance at exactly the observed times and grouping labels; refit both models by maximum likelihood for each simulated sample; and calculate . The upper-tail proportion, conventionally , estimates the p-value. Boundary fits with estimated must remain in the simulation distribution. A restricted-likelihood ratio and a correspondingly calibrated bootstrap are another option when the fixed-effect design is identical, but one must not mix ordinary and restricted likelihoods in a single statistic.
Past exam of the mathematics course of the University of Cambridge 2017 iii Paper 206 6 e Solution Created 2026-10-03 Updated 2026-10-05
Let have rows and let be the spatial covariance matrix with . Conditional on chosen covariance parameters, the best linear unbiased estimator of the drift coefficients is the generalized least squares estimatorThe printed procedure first estimates the linear drift from where those arguments are initial values, not the reported final estimates. Such semivariogram fitting typically minimizes a weighted sum of squared discrepancies between binned empirical values and model values; it is not automatically a joint Gaussian likelihood maximization. One can iterate: form residuals using the current drift, refit the covariance model, recompute the generalized least squares drift, and repeat until changes are small. The displayed commands themselves show a single sequence, not evidence that such iteration occurred.
depth ~ y, constructs a binned residual empirical semivariogram, fits a Gaussian semivariogram to it, and then uses the fitted covariance in generalized least squares trend estimation and universal kriging. The missing fit could be supplied asgauss.model2 <- fit.variogram(smvg2, vgm(0.5, "Gau", 1, 0))A likelihood-based alternative estimates the two blocks coherently fromSubstitute to obtain a profile log-likelihood, maximize over and with a positive-definite matrix , and recover the drift coefficients from the optimizing covariance. Restricted maximum likelihood adds the design determinant term and uses in place of in the normalizing term, up to a fixed-design constant. It can reduce bias from estimating the drift before the covariance. A zero fitted nugget is a legitimate boundary estimate, not proof that measurement error is absent. Estimate drift using covariance-weighted regression, and spatial dependence from drift-adjusted variation, with the two stages or likelihood explicitly distinguished.
Past exam of the mathematics course of the University of Cambridge 2022 ib Paper 3 18H a Solution Created 2026-09-24 Updated 2026-09-25
The log likelihood, up to constants, isDifferentiation gives the generalized least squares estimatorand maximizing over the scale givesThe divisor is appropriate for maximum likelihood rather than unbiased estimation.
Range restriction in singular-covariance spectral fitting requires a residual to lie in the subspace where the measurement model allows random error. Let with zero-mean error covariance . Every vector in the kernel of satisfiesso almost surely. Thus a permissible residual obeyswhere is the Moore-Penrose inverse. The matrix is the orthogonal projector onto the range of the covariance.
Only after enforcing this condition should a generalized least squares objective use . The quadratic form alone assigns zero penalty to the forbidden directions and therefore cannot impose the condition. Rank deficiency from deterministic spectral upsampling gives its physical spectral-grid origin through . Covariance propagation preserves those exact dependencies.[ref]
For an original idealized two-channel example, take , mean and observation . The second channel has no random error in this stipulated model, so the range condition forces . The remaining squared residual is one. Minimizing only the pseudoinverse quadratic would instead choose and objective zero, while predicting the impossible second-channel value six.
Real measurements need not contain a perfect channel: deterministic rebinning produces exact noise-free combinations of otherwise noisy channels. Additional physical noise in such a combination must be modelled if present. If covariance ranges change with the parameters, pseudoinverse fitting with a parameter-dependent covariance range requires enforcing the appropriate support for each candidate; a single fixed projection need not apply.
Raman detector-array spectra are recorded before numerical redistribution onto another spectral grid.[ref]
Femtosecond Broadband Stimulated Raman: A New Approach for High-Performance Vibrational Spectroscopy by David W. McCamant and others. Instrumentation, array detection; Data Collection, equation 4
Fitting the original independent measurement coordinates often makes these support restrictions explicit and avoids treating interpolation points as new observations.Rank deficiency from deterministic spectral upsampling occurs because a deterministic map cannot manufacture independent measurements. For and input covariance matrix ,When the output has more channels than the input, its covariance matrix is necessarily singular. Covariance of count-conserving rebinned Raman channels gives the conserved-count example and its full propagated uncertainty.[ref]
Using that parent's two-to-three channel map, the output always satisfies . If the input readings are and , the rebinned vector is . Recovering the original readings as and shows that the third displayed number is not a third independent observation.
For an original fitting example, suppose both input means equal an unknown and their independent measurement-error variances are known to be . Fitting in the original channels gives , with variance . A correct generalized least squares fit after rebinning gives the same result and uncertainty. It must use the singular covariance on its supported range, for example through the Moore-Penrose inverse, together with the exact linear constraints.
The range restriction in singular-covariance spectral fitting matters if a proposed output model fails those constraints. A residual along cannot arise from this rebinning model at all. Merely inserting a pseudoinverse into a quadratic objective would assign zero penalty to that impossible direction. Either enforce the range condition or supply a physically justified additional noise model; adding a small diagonal term solely to make the matrix invertible changes the statistical assumptions.
The motivating Raman apparatus measures detector-array spectra before any subsequent numerical grid change.[ref]
Femtosecond Broadband Stimulated Raman: A New Approach for High-Performance Vibrational Spectroscopy by David W. McCamant and others. Instrumentation, array detection and spectral calibration; Data Collection, equation 4
Working on the original grid often avoids this singular fitting problem while retaining the actual measurement information. RNA log-odds covariance 2026-11-05
RNA log-odds covariance describes correlated errors between transformed population estimates at different temperatures in an RNA melting log-odds plot. Shared RNA melting baselines can generate it even when individual detector readings are independent.
Let be unfolded fractions and . First-order propagation givesIn matrix form, with diagonal . This is the multivariate form of RNA melting log-odds uncertainty; the derivatives are evaluated at the estimated fractions.[ref]
A concrete shared-input model uses . If all folded-state predictions contain the same uncertain offset , thenThis component has one shared error direction. It is not equivalent to adding independent noise with variance to every observation. Other baseline parameters contribute through RNA baseline prediction covariance.
For an original simplified example, nine observations each have independent log-odds standard uncertainty0.03 and the same shared-offset contribution 0.01. The standard uncertainty of their mean isIncorrectly treating the shared term as independent would give . Repetition reduces the independent component while leaving this common offset contribution intact.
A model fit can use the joint covariance matrix in generalized least squares, with an objective of the form when that inverse exists. Alternatively, fit the underlying absorbance and baseline parameters jointly. Shared-error components alone may be singular, and near population endpoints the logarithmic linearization may fail. A precise straight-line fit does not independently validate the two-state RNA melting model or its baseline assumptions.
Tie-line phase-fraction fit 2026-11-05
A tie-line phase-fraction fit estimates how much of each liquid phase is needed to reproduce an overall composition from specified liquid-liquid tie line endpoints. If are endpoint mole fraction vectors and is the mole-based phase fraction of phase , the mass balance predictsThe endpoints must be supplied by equilibrium measurements or a suitable thermodynamic model; fitting does not establish that an arbitrary pair of compositions is an equilibrium tie line.[ref]
Tie-line in IUPAC Gold Book. Entry15367: endpoints are compositions of coexisting equilibrium phases
For a ternary liquid-liquid tie line, use two independent composition coordinates, such as the first two mole fractions, since the third is fixed by their sum. Let be these two measured coordinates, the corresponding phase- coordinates, the difference between phase- and phase- coordinates, and a positive-definite covariance matrix for . With exact endpoints, generalized least squares minimizesDifferentiation yields the unconstrained estimateThe second expression imposes the physical bounds. It follows because is a convex quadratic when the endpoints differ in the measured coordinates. The ternary tie-line consistency residual then records the remaining failure of the mass balance.
For an original hypothetical example, take endpoint compositions and , with overall measurement . Suppose the first two measured mole fractions have independent standard measurement uncertainties and , so . ThenThe predicted composition is , giving residual . The separate first- and second-component balances would suggest and ; the fit combines them according to their stated measurement uncertainties, rather than averaging those two numbers without regard to precision.
With Gaussian composition errors, fixed endpoints and an interior unconstrained fit, the tie-line phase-fraction uncertainty follows directly from the linear estimator:for this example. Endpoint uncertainty, a fit on a boundary, or a nearly vanishing endpoint difference needs additional treatment. In particular, all three normalized mole fractions must not be treated as independent observations with an invertible three-coordinate covariance matrix: their exact closure creates dependence. A small fitted residual is evidence of consistency with the specified mass balance and error model, not a substitute for testing chemical potential equality or phase stability.
Tie-line phase-fraction sensitivity 2026-11-05
Tie-line phase-fraction sensitivity measures how an inferred phase fraction changes when the overall composition or the measured liquid-liquid tie line endpoints change. A sensitivity coefficient is a local partial derivative of the inference formula, rather than an uncertainty by itself. It becomes an uncertainty contribution only after combination with the input covariance matrix.[ref]
Guide to the expression of uncertainty in measurement by JCGM. §§5.1.3–5.1.4, sensitivity coefficients and their experimental determination
For one composition coordinate of a binary mixture, write the lever rule aswhere are endpoint mole fractions, is the overall mole fraction and is the mole-based amount of phase . Differentiating the mass balance givesThus a measurement error in the overall composition and an equal measurement error in the phase- endpoint have different effects. The conditioning of tie-line phase-fraction inference deteriorates as the endpoint separation becomes small: the same composition measurement error then produces a larger change in the inferred phase fraction.
For an original example, take , , , giving . Increasing only by gives a first-order change . Direct substitution gives , a change of . For the closer endpoints , and balanced composition , the local sensitivity to the same endpoint error is ten times larger. This is a property of the inference geometry; it does not establish whether either hypothetical pair is in liquid-liquid equilibrium.
The multidimensional tie-line phase-fraction fit has an additional detail away from an exactly balanced composition. On independent mole fraction coordinates, hold a symmetric positive-definite matrix fixed and definewith distinct endpoints. Define row vectors and . Differentiating the generalized least squares normal equation givesThe residual correction in tie-line sensitivity vanishes when . Otherwise, moving an endpoint also rotates the fitted line relative to the nonzero ternary tie-line consistency residual. Omitting that contribution is an additional approximation. If itself changes with the measured inputs, its derivative must also enter the calculation.
These formulas describe the unconstrained fit locally. Imposing introduces boundary effects, and coincident endpoints make this inference undefined. A practical sensitivity check recomputes the fit after small positive and negative perturbations of one measured coordinate, keeping composition closure and the stated weight matrix. Agreement as the perturbation shrinks tests the derivative calculation; it does not validate the underlying equilibrium model.
Closer tie-line endpoints amplify composition errors; endpoint correlation changes the inferred phase-fraction uncertainty
. Original binary examples with phase fraction0.40. Left: exact endpoint perturbations at separations0.60 and0.06. Right: first-order standard uncertainty with uncertain endpoints and exact overall composition; common calibration of all three measurements is a different model. Details. Universal kriging Created 2026-10-05 Updated 2026-10-18
Universal kriging allows an unknown regression drift:The design matrix has full column rank, its entries and are known, and the regression coefficients are unknown. For example, rows and target row allow a linear spatial trend. Requiring to have zero bias for every gives . With known positive-definite matrix observation covariance matrix , the minimizing weights solvewhere is a vector of Lagrange multipliers. Equivalently, the generalized least squares estimate and prediction areThe first term is the estimated trend alone; the second predicts the correlated regression residuals. With , the error variance isThe last, nonnegative term accounts for estimating the specified drift coefficients. Merely inserting a fitted drift into simple kriging and keeping its known-mean error variance would omit this term. For a Gaussian random field, the same expressions arise by integrating over drift coefficients in a diffuse prior distribution limit; this is an additional inferential interpretation, not an assumption required by the linear optimization.[ref]
Gaussian Processes for Machine Learning by Carl Edward Rasmussen and Christopher K. I. Williams. §2.7, pp. 27–28, equations (2.38)–(2.42)
