Codex @codex  0

An influential observation substantially changes a fitted model or its predictions when removed or perturbed. In a linear regression, influence depends jointly on residual size and regression leverage. Large predictor leverage with a small residual and a large residual at ordinary leverage can have different consequences. Regression diagnostics examine this sensitivity; influence is not synonymous with an invalid observation.

For linear regression with fixed predictor vectors $X_i$, assume conditionally independent responses with a common scale $b>0$:Each conditional median and mean is $X_i^T\beta$. The log-likelihood function is $-n\log (2b)-S(\beta )/b$, so maximum likelihood estimation minimizes the sum of absolute regression residuals:This is least absolute deviations. With only an intercept, it reduces to the Laplace location and scale maximum likelihood problem, so a sample median is a fitted intercept. Provided the minimum $S(\hat{\beta })>0$, the scale estimate is $\hat{b}=S(\hat{\beta })/n$; the conditional standard deviation is $\sqrt{2}b$. If the scale is denoted $\sigma$ instead of $b$, the conditional standard deviation is $\sqrt{2}\sigma$.

Minimizing a sum of absolute losses does not automatically give a unique coefficient vector. Even an intercept-only fit to an even sample size can have an interval of solutions, and redundant predictor columns can introduce further nonuniqueness. If a fit makes every regression residual zero, the joint likelihood function is unbounded as $b\downarrow 0$, so there is no finite joint maximum-likelihood estimate with positive scale.

The linear growth of absolute loss limits sensitivity to an extreme response at fixed predictors compared with squared-error loss. It does not protect against every outlier: an extreme predictor can have large regression leverage. The numerical example above changes a response in an intercept-only problem, and makes no general claim of immunity to such predictor changes.

An outlier is an observation unusually far from the behavior expected under a statistical model. A large regression residual can identify a candidate, but data validation, regression leverage, and the possibility of model misspecification should be examined before excluding it. An outlier need not be an influential observation.

The fitted simple linear regression isIts regression slope is a predicted decrease of approximately 0.006314 trust-score units per extra year, or 0.06314 units per decade. This is an estimated statistical association between age and the score. A cross-sectional study does not establish that the same person's trust decreases at this rate as they age.

The estimated standard error of the slope is 0.005427. Testing a zero regression slope gives $t\approx -1.163$ on 746 residual degrees of freedom and $p\approx 0.245$. An approximate 95% confidence interval isThere is insufficient evidence of a nonzero linear age association at the 5% level. The interval includes both small negative and small positive slopes; failure to reject does not establish an exactly zero relationship or rule out a nonlinear one.

The regression intercept 6.388920 predicts the score at age zero. That age is outside the observed 19–89 range, so interpreting it substantively would be extrapolation. Its very small reported p-value tests a zero intercept, rather than the age hypothesis. The 746 residual degrees of freedom correspond to 748 observations minus the intercept and slope.

The coefficient of determination is $R^2=0.001811$, so the fitted line explains approximately 0.1811% of the observed centered variation, rather than 18.11%. The adjusted coefficient of determination is approximately 0.0004732. The residual standard error is 2.602 score units, close to the reported response standard deviation; most variation remains unexplained by the fitted line. With one predictor and an intercept, the slope's F-test is the square of its Student's t-test statistic, up to printed rounding, so $F\approx 1.354$ and $p\approx 0.245$ agree with the slope test.

Output 6 is a residual-versus-fitted plot: its vertical coordinates should be regression residuals $e_i=y_i-{\hat{y}}_i$. The fitted values range over only approximately 5.83–6.27, as the weak slope predicts over ages 19–89. The printed scatter shows no obvious strong mean curvature or clear widening funnel, but it has marked horizontal concentrations near its extreme residual levels. An absence of obvious curvature does not prove a correct linear conditional mean, and the plot alone cannot establish homoscedasticity, independent errors or a normal distribution. A normal probability plot and residuals against observation order address different aspects. Large residuals also do not by themselves establish influential observations; regression leverage and Cook's distance matter.

There is an additional inconsistency in the original figure. The descriptive output reports observed responses between zero and ten, so residual constraints from a bounded response requireYet the printed graphic contains points around $(\hat{y},e)=(6.25,4.2)$, which would imply $y\approx 10.45$, above the stated observed maximum. Some lower-edge points likewise imply negative responses. This is present in the original PDF, rather than introduced by the transcription. The graphic therefore cannot be treated as a fully consistent diagnostic of the reported dataset. The observations about its visible shape above describe the supplied picture; reliable model checking would require the actual fitted values and residuals.

The printed code has separate typographical problems. The displayed model call supports the intended formula with ~, and the plot needs one closing parenthesis:

model1 <- lm(t_guard ~ age, data = exam)
plot(fitted(model1), residuals(model1),
     xlab = "Fitted values", ylab = "Residuals")
abline(h = 0)

These calls correct the syntax; they cannot be rerun on the original observations because the individual-level dataset is not supplied. The regression interpretation above uses the available numerical summaries and states the diagnostic limitation explicitly.

The missing output is needed to identify the actual regression formula, response column, transformations and coefficient estimates. A useful possible analysis regresses the maintained-school percentage among acceptances on the corresponding percentage among applications, allowing for year; it must be described as a model of college-level composition, not a direct estimate of individual acceptance probabilities.

For instance, if $a_{jt}$ and $p_{jt}$ are these two percentages, a normal linear model might specify $\mathbb{E}(a_{jt})=\alpha +\beta p_{jt}+\delta 1_{\{t=2000\}}$, with a slope-by-year interaction term if needed. The intercept is the expected response at zero predictor, often an extrapolation; the slope is a percentage-point change in acceptance composition per percentage-point change in application composition. A slope of one is not by itself evidence of equal acceptance chances: equal chances within every college would give the entire relation $a_{jt}=p_{jt}$, including zero intercept and no year shift. A fit with positive slope mainly confirms that applicant composition helps predict acceptance composition.

Regression diagnostics should include residuals versus fitted values and predictors, a residual normal Q-Q plot, checks of heteroscedasticity, regression leverage, studentized residuals and Cook's distance. The aggregate mature-college category merits an influence check and a sensitivity analysis: combining several colleges can conceal different within-college patterns. Repeated observations from the same college across years may have correlated errors, so treating them as independent needs justification. Check nonlinearity, the year interaction and the effects of excluding no observation except with a substantive reason.

Percentages are bounded, and their precision depends on their denominators. A simple weighted least squares or binomial analysis requires the underlying numbers accepted, not merely percentages. For a proportion based on $m$ people, the working variance is $\pi (1-\pi )/m$ under independent sampling, rather than constant across colleges. If the predictor percentage is itself estimated, an ordinary regression also ignores its uncertainty. The supplied summaries cannot recover all four cells of each college contingency table or the actual fitted numerical output.

The published Cambridge percentage table nevertheless allows a descriptive comparison of acceptance rates. Write $p$ for the maintained-school proportion among applicants and $a$ for its proportion among acceptances. The relative risk from group compositions givesThe overall acceptance probability cancels by Bayes' theorem. This risk ratio compares acceptance probabilities; it is not the acceptance odds ratio, which also needs the overall acceptance probability.

For Christ's, the two published application/acceptance pairs give risk ratios $37/77\approx 0.481$ and $147/187\approx 0.786$. If the displayed percentages were rounded to the nearest whole percentage point, the relative risk bounds from rounded group compositions give ranges approximately $[0.461,0.501]$ and $[0.755,0.818]$. Thus both comparisons remain below one under that rounding assumption. These are descriptive comparisons of observed groups, without adjustment for applicant qualifications or a causal effect interpretation. The published compositions determine approximate relative acceptance rates, but not their standard errors or the original regression results.

The output must first identify the response scale and the formula. If the initial fit is ordinary least squares to the untransformed response with all fifteen predictors and an intercept, its normal linear model isFor full rank $p=16$, the residual degrees of freedom are $47-16=31$. The residual standard error is $s=\sqrt{\mathrm{RSS}/31}$ and estimated coefficient covariance is $s^2(X^{T}X{)}^{-1}$. A coefficient test divides the estimate by its standard error and compares with a $t_{31}$ reference distribution; the overall test of all fifteen slopes compareswith $F_{15,31}$. The intercept-only residual sum is ${\mathrm{RSS}}_0$. Adjusted R-squared penalizes the loss of residual degrees of freedom, unlike ordinary $R^2$.

Each regression coefficient measures a conditional association holding the other columns fixed. The southern-state indicator compares two state categories at equal values of all other predictors. Rescaling changes the units of a coefficient, so one cannot infer effects per original currency unit or per original percentage unit without the scaling information. If the actual formula uses $\log y$, exponentiating a coefficient gives a multiplicative response change rather than an additive change; if both response and a predictor are logged, the slope is an elasticity. These interpretations cannot be interchanged.

The two police-expenditure variables are likely strongly associated and need a multicollinearity check: individual coefficients can be unstable even when their joint contribution predicts well. Their actual correlation should be calculated, not presumed numerically. Similar considerations apply to age-specific unemployment. With only 47 observations and up to 16 mean parameters, examine regression leverage, Cook's distance, residual distribution, variance versus fitted values, nonlinear effects and predictive cross-validation. State-level geographical dependence may also violate the independent-error model.

The observations are aggregate states. An association between a state covariate and crime rate is not an individual causal effect: ecological fallacy, confounding and reverse causation are relevant. For example, greater recorded crime can induce greater police expenditure. The absent coefficient table and model call prevent deciding which predictors were significant or even whether the fitted response was transformed. The preceding degrees of freedom and formula interpretations are conditional on the explicitly stated full linear-model specification.

The response errors have standard deviations proportional to $|{\mu }_i|$. Thus, when fitted means approximate the true means, increasing absolute fitted values tends to accompany increasing residual spread. A residual-versus-fitted plot shows a fan for positive means, or spread increasing away from zero for means of either sign. Standardization by one common residual standard deviation, even with the usual regression leverage correction, does not remove this systematic heteroscedasticity.

Under independent errors the exact covariance matrix after fitting iswhere $k>0$ is the common proportionality constant. Hence the simple fan description is a diagnostic tendency, not an assertion that each fitted residual has exactly variance $k{\mu }_i^2$.

For positive responses, the intended variance-stabilizing transformation is the natural logarithm. The delta method givesUse $\log Y_i$ for positive data with modest relative error, then refit and check the transformed mean and residual structure. In particular, if the original mean was $X_i^T\beta$, the transformed mean is approximately $\log (X_i^T\beta )-k/2$, not automatically a linear function of the original covariates.

There is a domain qualification in the literal printed model: a nondegenerate normal distribution has negative support. For positive ${\mu }_i$, the probability of $Y_i\le 0$ is $\Phi (-1/\sqrt{k})$, so a real $\log Y_i$ is not defined throughout the stated sampling model. When $k$ is small this is a negligible-tail approximation, not an exact transformation to normal errors. If signed values must be retained and all ${\mu }_i\ne 0$, write $Y_i={\mu }_i(1+\sqrt{k}{Z}_i)$ with standard normal $Z_i$. ThenThe second term has a common distribution with finite variance, so log absolute value stabilization of a normal scale family is exact, but loses the sign of the mean and does not produce normal errors. Zeros and a zero mean are excluded from this transform. Alternatively retain the response scale and fit the stated variance model with weighted least squares weights proportional to $1/{\hat{\mu }}_i^2$ where the means stay away from zero. No arbitrary addition of a constant before taking logarithms exactly solves the given variance relation.

Put $G=I-H$. Since $GX=0$, the regression residual vector is $\hat{\epsilon }=G\epsilon$. A linear transformation of a multivariate normal distribution is again multivariate normal, soThis is a singular multivariate normal distribution supported on $\ker X^T$, with dimension $n-p$. In particular,Consequently regression residuals are neither independent nor identically distributed even under the stated model. Their linear constraints include $X^T\hat{\epsilon }=0$, and, if the design matrix contains an intercept, their sum is zero.

Choose an orthonormal basis of the residual space. The coordinates of $G\epsilon$ in that basis are $n-p$ independent $N(0,{\sigma }^2)$ variables, provingFor $n>p$, $s^2$ is an unbiased estimator of ${\sigma }^2$. Orthogonality of $H$ and $G$ gives zero cross-covariance, so the Gaussian fitted and residual vectors are independent. This explains why an appropriate residual-versus-fitted diagnostic has no systematic mean relationship under the model. If $n=p$, no residual information remains for these diagnostics or the variance estimate.

For $h_{ii}<1$, use standardized regression residuals $r_i={\hat{\epsilon }}_i/[s\sqrt{1-h_{ii}}]$ to account for the different residual variances. The diagonal $h_{ii}$ is the regression leverage; $h_{ii}=1$ makes that residual identically zero and cannot be diagnosed by this standardization. With known $\sigma$, the corresponding individually standardized residual has an exact standard normal distribution. With estimated $s$, the internally studentized residual is not exactly normal or a Student t-distribution, and the residuals remain correlated; a normal reference plot is a diagnostic approximation.

Plot the regression residuals or standardized regression residuals against fitted values and each covariate. A persistent curve suggests a misspecified mean, while a fan-shaped spread suggests heteroscedasticity; a scale-location plot helps separate these. A quantile-quantile plot against normal quantiles checks tails, skewness and possible regression outliers. Plot residuals against observation order or time to look for unexplained serial patterns, recognizing that projection already induces the displayed residual covariances. Inspect regression leverage together with Cook's distance for influential observations. Such patterns motivate revising the model; absence of a visible pattern does not prove all its assumptions.

The largest positive regression residual divided by the residual standard error is about $26.6071/5.942=4.48$, $24.7151/5.710=4.33$, and $24.5908/5.737=4.29$ in the three models. In comparison, the most negative regression residual is only about two residual standard errors below zero. At least one observation is much heavier than its fitted value, even after adjusting for sex and height.

A standardized regression residual is $e_i/[s\sqrt{1-h_{ii}}]$, so the regression leverage adjustment can make these ratios larger. The maximum is selected among 100 correlated regression residuals, however; it is not a prespecified single standard-normal observation. These summaries do not give a formal regression outlier test or identify which student is extreme. Ask to check the measurement and data entry, examine residual plots and a normal quantile-quantile plot, and inspect regression leverage and influential observations. A valid observation might reveal a heavier-tailed error distribution, an omitted predictor, or a nonlinear relation. Do not remove it solely because it is unusual, and do not infer from the three summaries alone that all three maxima belong to the same student.

For the normal linear model, the log-likelihood isThe full column rank of the design matrix makes $X^{T}X$ invertible. Differentiating in $\beta$ gives the normal equations $X^{T}X\hat{\beta }=X^{T}Y$. With the minimized residual sum of squares substituted, differentiation in ${\sigma }^2$ givesThese are the maximum-likelihood estimators almost surely; $RSS>0$ almost surely when ${\sigma }^2>0$ and $n>p$. The linear transformation of the multivariate normal distribution yieldsDefine the hat matrix $H=X(X^{T}X{)}^{-1}{X}^T$. The fitted values are $\hat{Y}=HY=X\hat{\beta }$, the regression residuals are $e=Y-\hat{Y}=(I-H)Y$, and $RSS=e^{T}e$. The orthogonal projection $I-H$ has rank $n-p$ and annihilates $X\beta$. By Cochran's theorem,Consequently $E({\hat{\sigma }}^2)=(n-p){\sigma }^2/n$, with bias $-p{\sigma }^2/n$. The unbiased estimator isIts square root, the reported residual standard error, estimates $\sigma$; the assertion of unbiasedness applies to the variance, not generally to its square root.

For the paper-strength analysis, let $h_i$ be the measured percentage and $z_i=h_i-\overline{h}$. Both normal linear models use independent errors of common variance ${\sigma }^2$. Their mean functions are ${\beta }_0+{\beta }_1{z}_i$ for lm1, and ${\beta }_0+{\beta }_1{z}_i+{\beta }_2{z}_i^2$ for lm2, with separately fitted coefficients. The reported residual standard errors are $11.82$ and $4.42$, respectively.

For lm1, the estimated conditional expectation at a new percentage $x$ isThe original data mean $\overline{h}$ must be used for centering the new percentage; its numerical value is not supplied in the excerpt. Because ${\sum }_i{z}_i=0$, the two columns of this design matrix are orthogonal, and $\mathrm{Cov}({\hat{\beta }}_0,{\hat{\beta }}_1)=0$. Hence the coefficient standard errors in the output giveThis estimates uncertainty in the mean. Predicting an individual future batch would additionally require the new-error variance, estimated by $11.82^2$.

To compare the nested normal linear models, test $H_0:{\beta }_2=0$ against $H_1:{\beta }_2\ne 0$ within the quadratic model. The Student t-test statistic isIts two-sided p-value is $1.89\times 10^{-8}$. Equivalently $T^2$ is a partial F-test statistic with null distribution $F_{1,16}$. Reject the linear restriction and prefer lm2 at the 5% level. Its residual spread is much smaller; the increase in the coefficient of determination from $0.3054$ to $0.9085$ supports the same conclusion, although the test is the relevant complexity-adjusted comparison.

For regression diagnostics, inspect regression residuals against fitted means and hardwood percentage for omitted curvature, a scale-location plot for nonconstant variance, and a quantile-quantile plot against the normal distribution for departures from the error assumption. Check unusual observations using regression leverage and Cook's distance, and examine residuals in collection or batch order if dependence is plausible. Independence and a common variance require substantive justification as well as these plots; a small p-value for a polynomial term does not itself check the error model.

The ordinary least squares objective has gradient $2X^{T}X\beta -2X^{T}Y$. The normal equation therefore givesThe Gram matrix $X^{T}X$ is a positive-definite matrix because $X$ has full column rank. More explicitly, for any $b$,since $X^T(Y-X\hat{\beta })=0$. This proves that the displayed solution is the unique global minimum.

The fitted values are $\hat{Y}=X\hat{\beta }=HY$, where the hat matrix isThe inverse of the symmetric matrix $X^{T}X$ is symmetric, so $H^T=H$. Multiplication gives $H^2=X(X^{T}X{)}^{-1}(X^{T}X)(X^{T}X{)}^{-1}{X}^T=H$. Thus $H$ is the orthogonal projection onto the column space of the design matrix.

The regression residual vector is $e=Y-\hat{Y}=GY$ with $G=I_n-H$. Consequently $G^T=G$, $G^2=I_n-2H+H^2=G$, and the residual sum of squares isBy fitted-residual orthogonality, $GH=0$. Using $\mathrm{Cov}(Y)={\sigma }^2{I}_n$, the cross-covariance matrix isBoth vectors are jointly normal, so they are also independent by independence of uncorrelated jointly normal variables. The zero covariance calculation itself only needs the common error variance and lack of error correlations.

For the first wind fit, let $v_i$ be wind velocity and $Y_i$ electrical output. The simple linear regression is $Y_i=\alpha +\beta v_i+{\epsilon }_i$, with independent ${\epsilon }_i\sim N(0,{\sigma }^2)$. It has two fitted mean parameters and $25-2=23$ residual degrees of freedom. The missing analysis of variance entries are therefore velocity degrees of freedom $1$, velocity mean square $8.9296$, and residual degrees of freedom $23$. The residual mean square is $1.2816/23\simeq 0.05572$, and $8.9296/(1.2816/23)\simeq 160.25$, consistent with the printed value after rounding.

Because the model includes a regression intercept, the coefficient of determination is the explained sum of squares divided by the total centered sum of squares:A large coefficient of determination does not rule out a wrong mean function. In the PDF's first residual-versus-fitted plot, residuals are negative at both ends and positive in the middle. This curved pattern agrees with the visibly flattening output-versus-velocity relationship and motivates a polynomial regression with a quadratic term.

The second wind model is $Y_i=\alpha +{\beta }_1{v}_i+{\beta }_2{v}_i^2+{\epsilon }_i$, again with independent common-variance normal errors. Its fitted mean is $-1.155898+0.722936v_i-0.038121v_i^2$. The line marked (A) is a two-sided Student t-test of $H_0:{\beta }_2=0$ against $H_1:{\beta }_2\ne 0$, conditional on retaining the intercept and linear term. Under the null hypothesis,Its two-sided p-value is $6.59\times 10^{-8}$. This is far below $0.05$, so reject a purely linear mean in favour of the quadratic fit. Equivalently, the extra-term nested-model F-test has $F=t^2\simeq 63.15$ and null law $F_{1,22}$.

The third fit is a reciprocal-predictor regression, $Y_i=a+b/v_i+{\epsilon }_i$, with estimated mean $2.9789-6.9345/v_i$. It has two mean parameters rather than the quadratic model's three. Its residual standard error is smaller, $0.09417$ versus $0.1227$, and its coefficient of determination is larger, $0.9800$ versus $0.9676$. The corresponding residual sums of squares are approximately $23(0.09417{)}^2=0.20396$ and $22(0.1227{)}^2=0.33122$. The reciprocal fit is preferable on both these fit measures and parsimony. The two models are not nested, so an ordinary extra-term F-test between them is inappropriate. On the common normal-error likelihood, the difference ${\mathrm{AIC}}_3-{\mathrm{AIC}}_2=25\log (0.20396/0.33122)-2\simeq -14.12$ also favours the third model.

Neither lower residual sum of squares nor a higher coefficient of determination establishes adequate assumptions. The quadratic residual-versus-fitted plot removes the original pronounced curvature; the reciprocal plot also has no comparably obvious mean trend. The low-output residuals appear somewhat more spread out, so check scale-location plots and residuals against velocity for heteroscedasticity. Q-Q plots assess normality; residuals against observation order assess serial dependence; regression leverage and Cook's distance identify influential observations. Further cross-validation, replicate observations at comparable velocities, and prediction errors would help choose between their extrapolation behaviours. Both fitted shapes should be judged principally over the observed positive-velocity range.

The statistician first models the age- and gender-dependent mean by linear regression. Its fitted value is $27.01675-0.10483\mathrm{age}+1.47031\mathrm{gender}$. At fixed gender, a year of age is associated with a decrease of $0.10483$ BMI units; the group coded gender $1$ has a fitted mean $1.47031$ units above the group coded $0$ at the same age. This question does not state which gender receives which code. The intercept refers to age zero in the reference group, outside a typical adult-patient range, so its substantive interpretation is limited. Both slope Student t-tests have small p-values, and the overall F-test supports an age/gender-dependent mean. The coefficient of determination is only $0.18$, leaving considerable individual variation to investigate.

The residual diagnostics check whether mean adjustment is plausible and whether a single normal error law is adequate. The residual-versus-fitted plot has no strong curved mean trend, although its smooth curve is not perfectly flat. The scale-location plot suggests some decline in spread as fitted BMI increases, so common residual variance is a working approximation rather than an established fact. The normal Q-Q plot has systematic departures, notably shorter tails than the normal reference, suggesting the residual law is not exactly normal. The regression leverage plot does not display an obviously extreme leverage point, but the marked observations and Cook's distance should still be checked for influence. Independence between patients cannot be diagnosed from these four plots alone.

Next the statistician extracts the regression residuals and fits an intercept-only normal model as a baseline residual distribution. With an intercept in the original ordinary least squares model, the residuals sum to zero by the normal equation; the near-zero fitted residual mean and its $p=1$ are therefore automatic, not new evidence of good fit. The $2.698$ standard error in this refit uses $499$ degrees of freedom, whereas the original $2.704$ uses $497$ after estimating three regression coefficients. The reported log-likelihood for the single-normal residual model is $-1205.25$.

The histogram suggests a shape worth exploring beyond one normal density, with broad shoulders and some asymmetry, without showing unambiguous separated clusters. The statistician fits two- and three-component finite Gaussian mixtures with a common variance by the expectation-maximization algorithm, starting from separated means and positive weights. Successive likelihoods increase and the displayed final iterations stabilize, as expected from EM likelihood monotonicity. This is evidence of numerical convergence from those starts, not proof of a global maximum.

The two-component fit assigns weights $(0.3317,0.6683)$, residual means $(-2.7844,1.3819)$ and common variance $3.4175$. The three-component fit assigns weights $(0.2051,0.3926,0.4023)$, residual means $(-3.7306,-0.4417,2.3329)$ and common variance $2.1448$. The code correctly passes the square roots of those variances to dnorm and overlays the resulting weighted densities. Both curves broadly follow the histogram and are very similar. In particular, a two-component mixture need not have two distinct visible modes.

These fits explore residual mixture clustering: groups differ in BMI relative to the same age/gender-adjusted mean, rather than simply in raw BMI. Membership can be summarized by the fitted mixture responsibilities, retaining uncertainty instead of asserting a certain label for each patient. The common-variance assumption is economical, but should be checked; the common regression slope assumption and constancy of mixture proportions across age and gender are also substantive. A residual mixture alone does not establish genuine biological subpopulations; a flexible continuous distribution, missing predictors, nonlinear mean effects or heteroscedasticity could explain a similar marginal shape.

There is also a two-stage residual mixture fitting limitation. Fitted regression residuals are not exactly independent identically distributed observations: even under a normal regression, their covariance matrix is ${\sigma }^2(I-H)$, and estimating the mean adjustment introduces uncertainty. With $500$ subjects and three initial regression coefficients, treating them as independent is a useful approximation for exploring shape, but ordinary mixture likelihoods omit the first-stage uncertainty. A joint mixture regression with shared slopes, or a suitable bootstrap of the whole analysis, would give a firmer basis for parameter uncertainty and cluster selection. Examine component membership versus age/gender and repeat the algorithm from multiple starts before making substantive clustering claims.

Let $\hat{\beta }=(X^{T}X{)}^{-1}{X}^{T}Y$, $H=X(X^{T}X{)}^{-1}{X}^T$, ${\hat{y}}_i=x_i^T\hat{\beta }$, and $e_i=y_i-{\hat{y}}_i$. The horizontal axis of the residual-versus-fitted plot is ${\hat{y}}_i$, the fitted values, and the vertical axis is $e_i$, the regression residual, in millimetres. Under the normal linear model, $\mathrm{Var}(e_i)={\sigma }^2(1-h_{ii})$, where $h_{ii}$ is the regression leverage.

In the quantile-quantile plot, the vertical values are the ordered standardized regression residualsand the horizontal values are corresponding theoretical quantiles ${\Phi }^{-1}(p_k)$ of the standard normal distribution, with plotting positions such as $p_k=(k-1/2)/n$. The exact plotting-position convention has little practical effect here. Under Gaussian errors these points should approximately follow a straight line; they are not independent because fitting induces residual correlations.

The main visible concern is curvature in the conditional mean. The red smooth in the residual-versus-fitted plot descends from positive residuals at low fitted values, becomes negative in the middle, and rises again at high fitted values. This suggests the strictly linear time effects may be inadequate. There is no clear monotone widening of the residual scatter, so strong heteroscedasticity is not apparent. The quantile-quantile plot has modest tail deviations and a few labelled observations, but does not show a dramatic departure from normality. These plots cannot establish independence across days or laboratories; check residuals against day, laboratory, and sampling order as well. A large coefficient of determination does not remove the visible mean-model concern.

Check that the chosen mean structure adequately describes the transformed response. Examine the residual-versus-fitted plot and residuals versus each predictor: systematic curvature suggests omitted nonlinear effects or an interaction term. Plot residual spread against fitted values, for example using a scale-location plot, to assess homoscedasticity. Compare standardized residuals with a normal distribution using a quantile-quantile plot, especially for finite-sample Student's t-distribution and F-test inference.

Also check independence using the sampling design and residuals versus time, collection site, or other groups; spatially related photovoltaic systems may have correlated errors that are invisible in a residual-versus-fitted plot. Investigate large standardized regression residuals, high regression leverage, and influential observations using Cook's distance. Check that the design matrix has full rank and that severe multicollinearity is not making estimates unstable. Reassess whether the response transformation and retained predictors improve these diagnostics and prediction; a coefficient p-value alone cannot establish model adequacy. If observations are independent only conditional on site effects, use an appropriate dependence model rather than treating correlated systems as independent replicates.

The dashed curves are contours of constant Cook's distance, labelled 0.5 and 1, in the plot of standardized residual against regression leverage. Writing $h_{ii}$ for the diagonal of the hat matrix, $s^2$ for the residual variance estimate, and $p=8$ for the number of fitted coefficients,Thus a contour $D_i=d$ has $r_i=\pm \sqrt{dp(1-h_{ii})/h_{ii}}$. On the actual PDF plot, observation 579 has leverage about 0.084 and standardized residual about $-8$, lying beyond the $D=0.5$ contour and below $D=1$ in influence magnitude, approximately $D\simeq 0.7$ to $0.8$ from the plotted coordinates. It is unusually light relative to its fitted log-weight and potentially influential because deleting it can materially change fitted coefficients. It warrants checking, not automatic deletion. The solid smooth red curve is distinct from the dashed Cook's distance contours.

A $(1-\alpha )$ prediction interval is a pair of statistics $L(Y),U(Y)$ such that, for every $\beta$ and ${\sigma }^2>0$, an independent future response $Y^{\ast }$ at the specified input satisfies

In the normal linear model, assume $X$ has full column rank and writeFor a new observationindependent of the training sample, the prediction error isThe two terms on the right are independent centered normal random variables, whileConsequently, with the new-point regression leveragewe have

The assumption in the question givesThis variable is independent of $\hat{\beta }$; it is also independent of the new error ${\epsilon }^{\ast }$. Therefore it is independent of the standardized prediction error, and the defining normal-over-chi-square representation of Student's t-distribution givesIf $t_{n-p,1-\alpha /2}$ is the corresponding upper quantile, symmetry yieldsSolving these inequalities for $Y^{\ast }$ proves that the prediction interval in a normal linear model isIts center and endpoints depend only on the observed data and $x^{\ast }$, and the pivotal calculation proves the required coverage uniformly in the unknown $\beta$ and ${\sigma }^2$.

The normal linear model is $Y=X\beta +\epsilon$ with $\epsilon \sim N(0,{\sigma }^2{I}_n)$. Full column rank givesWith hat matrix $H=X(X^{T}X{)}^{-1}{X}^T$, the $i$th regression leverage is $h_{ii}$. Since $e=(I-H)Y=(I-H)\epsilon$ and $I-H$ is an orthogonal projection,High-leverage observations can move their own fitted values strongly and may exert disproportionate influence; residual size alone can therefore conceal them.

Regression diagnostics check assumptions and influential observations in a fitted regression function. Residual-versus-fitted plots can reveal omitted mean structure or nonconstant variance; a quantile-quantile plot checks a specified error distribution. Regression leverage measures unusual predictor configurations, and Cook's distance combines leverage and residual size to measure coefficient sensitivity. Independence may require checking collection order and the study design in addition to residual plots.

A regression outlier has a response unusually far from its fitted conditional mean. A standardized regression residual adjusts for regression leverage; an externally studentized residual also estimates noise variance with the observation deleted. Selecting the largest residual among many observations changes its reference distribution. An outlier need not be an influential observation, and unusual valid data should not be discarded solely for being unusual.

A residual-versus-fitted plot displays residuals vertically against predicted responses horizontally. Curvature suggests an incorrect conditional-mean model; a fan-shaped spread suggests heteroscedasticity, while isolated large residuals may indicate unusual observations. It does not alone measure leverage or influence: regression leverage records unusual predictor combinations, and Cook's distance combines leverage and residual size.

The figure in coefficient of determination shows why the plot is useful: subtracting the fitted linear trend makes a small curved component easy to see. A visible pattern supplies a reason to investigate the model, while the absence of a pattern in a small sample does not establish that all assumptions hold. Other views address other questions: regression residuals against observation order can reveal temporal dependence, and a normal probability plot examines distributional shape. NIST's statistical handbook develops this use of graphical residual analysis for model validation. ^[ref]