Standardized regression residual Created 2026-10-05 Updated 2026-11-05
In a normal linear model, the th regression residual has variance , where is a diagonal entry of the hat matrix. Replacing by the residual standard deviation gives the standardized residual used in the normal Q-Q plot of R's linear-model diagnostic plots.
Sum of squares 2026-10-13
A sum of squares adds the squares of numbers for specified real numbers . It is nonnegative and vanishes exactly when every term is zero. For the coordinates of a vector, it is the squared Euclidean norm; for regression residuals, it gives the residual sum of squares minimized by ordinary least squares.
Tensile modulus experiment Created 2026-10-18 Updated 2026-10-20
MIT's Spring 2004 laboratory assignment Properties of Heat Treated Steel gives a concrete tensile testing protocol. Record transducer channels and calibration factors; measure specimen diameter and length, and the extensometer gauge length with its locking pin inserted. Zero the load cell, pin the threaded grips into their adapters, and attach the extensometer centrally. Apply the specified seating load, record actuator position, then acquire readings every second while moving at about . The handout derives engineering stress from load cell force, engineering strain from extensometer deformation, and Young's modulus by linear regression below the proportional limit; it warns that actuator-derived strain includes apparatus deformation.[ref]
Properties of Heat Treated Steel by Massachusetts Institute of Technology. Laboratory assignment 10, printed pp. 2–3: Procedure for Tension Tests; Calculations
For a modulus-focused measurement, select a load window after seating, hold temperature and loading rate consistent, and record force, local extension and crosshead displacement together. Fit within that window and calculate . Inspect the regression residuals for curvature; repeat at a lower maximum load and include unloading to check elastic deformation. These are diagnostic additions to the cited protocol, whose full test continues through failure. The handout's seating load is specific to its specimens and apparatus, rather than a universal preload.
An extensometer measures the change between its contact points; crosshead displacement measures travel of the machine's moving member. The manufacturer Instron explicitly distinguishes local strain from travel that includes machine deformation and machine compliance.[ref]
Static Axial Clip-On Extensometers by Instron. FAQs: How does a clip-on extensometer work?; When should I use a clip-on extensometer instead of crosshead displacement?
The gauge length can be shorter than the uniform length between grips. Thus even a perfectly rigid machine generally has : the local instrument sees only its own segment. Do not divide one channel's extension by the other channel's reference length.Grips must transmit axial force without slip. Misalignment adds bending moment and makes opposite sides experience different axial strains; an instrument on just one side may then report a biased modulus. A changing extension offset, hysteresis between loading and unloading, or load-dependent fitted slope is evidence to investigate seating, slip or material response. A straight line over a short interval alone does not identify which is responsible.
Training error Created 2026-09-24 Updated 2026-10-14
Training error is the average loss function evaluated on the observations used to fit the prediction rule. It is the empirical risk of the fitted rule on its own training data. With squared-error loss, it is the average squared regression residual, measuring fit to observed responses rather than performance on fresh responses. Flexible fitting usually makes it optimistically biased for prediction error, as quantified by prediction optimism.
For a reproducible polynomial regression example, take , , and the known noiseless target . Construct responses with fixed disturbancesOrdinary least squares over polynomials of polynomial degree at most 1, 2, and 7 gives training errors , , and . The monotone improvement follows because these candidate families are nested: every smaller-degree fit remains available in the larger family. This argument concerns minimum training error, not generalization, and assumes the fitting mathematical optimization is solved. Degrees 2 and 7 fitted instead to the exact noiseless responses have mathematically zero training error; the script obtains approximately and from rounding. Extra flexibility alone does not force overfitting.
Two-length machine compliance separation 2026-10-18
Prepare two uniform specimens with the same material, direction and , but different deforming lengths . Keep grip engagement, adapters, loading window and measurement configuration equivalent. Determine separately by linear regression with its own displacement intercept. Subtracting the two series equations eliminates the common machine compliance:This is an inverse problem: one length supplies only one equation for two unknowns, while two distinct lengths separate the length-dependent specimen contribution from the constant contribution. In a graph of compliance against length, the slope is and the vertical intercept is . It is the compliance, not , that is linear in .
The dimensional analysis is revealing: , so . For , the ideal model requires . A negative fitted challenges the common-intercept assumptions or the measurements. Two lengths determine two parameters exactly, so they cannot test the assumed straight line; three or more lengths allow regression residuals to reveal departures. Nearly equal lengths amplify slope errors through the small difference . If the actual apparatus compliances differ by , the inferred length slope becomes .
Two-stage residual mixture fitting 2026-10-06
Fitting a mixture model to regression residuals treats the fitted mean adjustment as fixed. Even with independent homoscedastic errors, ordinary least-squares residuals have covariance matrix and satisfy fitted linear constraints, so they are not exactly independent observations. A joint mixture regression with shared slopes or a bootstrap of the whole fitting process better accounts for the first-stage estimation uncertainty.
Universal kriging Created 2026-10-05 Updated 2026-10-18
Universal kriging allows an unknown regression drift:The design matrix has full column rank, its entries and are known, and the regression coefficients are unknown. For example, rows and target row allow a linear spatial trend. Requiring to have zero bias for every gives . With known positive-definite matrix observation covariance matrix , the minimizing weights solvewhere is a vector of Lagrange multipliers. Equivalently, the generalized least squares estimate and prediction areThe first term is the estimated trend alone; the second predicts the correlated regression residuals. With , the error variance isThe last, nonnegative term accounts for estimating the specified drift coefficients. Merely inserting a fitted drift into simple kriging and keeping its known-mean error variance would omit this term. For a Gaussian random field, the same expressions arise by integrating over drift coefficients in a diffuse prior distribution limit; this is an additional inferential interpretation, not an assumption required by the linear optimization.[ref]
Gaussian Processes for Machine Learning by Carl Edward Rasmussen and Christopher K. I. Williams. §2.7, pp. 27–28, equations (2.38)–(2.42)
