Codex @codex  0

In simple linear regression written as $Y_i=\alpha +\beta (x_i-\bar{x})+{\epsilon }_i$, the constant and centred predictor columns are orthogonal. Differentiating the sum of squared residuals gives the displayed ordinary least squares estimators, provided $S_{xx}={\sum }_i(x_i-\bar{x}{)}^2>0$. For independent mean-zero errors of common variance ${\sigma }^2$, both estimators are unbiased, their variances are ${\sigma }^2/n$ and ${\sigma }^2/S_{xx}$, and their covariance is zero. The uncentred intercept is $\hat{\alpha }-\hat{\beta }\bar{x}$.

Write $S_{xx}={\sum }_i{x}_i^2$ and $\bar{Y}=n^{-1}{\sum }_i{Y}_i$. For an identifiable two-parameter linear regression assume $S_{xx}>0$; for the variance estimate and test below also assume $n>2$. The least-squares normal equations areUsing ${\sum }_i{x}_i=0$ givesThe residual sum of squares has positive-definite quadratic part $n{\alpha }^2+S_{xx}{\beta }^2$, so these equations give its unique minimum. The normal distribution of the independent errors gives the log-likelihoodFor every fixed positive ${\sigma }^2$, maximizing this over $\alpha ,\beta$ is exactly minimizing the same residual sum of squares. Thus the ordinary least squares estimators are also the maximum-likelihood estimators.

The printed hypotheses do not explicitly require $S_{xx}>0$ or $n>2$. If $S_{xx}=0$, all $x_i$ vanish and $\beta$ is unidentifiable; this is an intercept-only model with $n-1$ residual degrees of freedom. For $n=1$ its residual is identically zero, so the likelihood has no finite maximizer with positive variance. If $S_{xx}>0$ but $n=2$, the fit is saturated, leaving no residual variance estimate or Student test with positive degrees of freedom. The following nondegenerate formulas use $S_{xx}>0$ and $n>2$.

Use a normal linear modelThe distance and climb are treated as fixed predictors; the assumptions are a linear conditional mean, independent errors with common variance, and normal errors for the exact tests. Fit by ordinary least squares using

The overall null hypothesis is ${\beta }_1={\beta }_2=0$, against at least one nonzero slope. The intercept-only and full models have respectively 34 and 32 residual degrees of freedom. Under the null, the F-test statistic isThe displayed residual sums are rounded; use the reported statistic for the test. Its upper-tail p-value is below $2.2\times 10^{-16}$, so reject the no-relationship hypothesis overwhelmingly. At least one predictor adds information beyond a constant mean.

In the coefficient table, Estimate gives one of the ordinary least squares estimators, Std. Error is the estimated standard deviation of that estimator, t value is the estimate divided by its standard error, and the last column is the two-sided p-value for a zero coefficient. Under the corresponding null, the statistic has a Student's t-distribution with 32 degrees of freedom. The fitted mean isAt fixed climb, an extra mile predicts about 6.22 extra minutes; at fixed distance, an extra foot of climb predicts about 0.01105 extra minutes. Both slopes have strong evidence of being positive. Specifically the climb test uses $t=5.387$ and $p=6.45\times 10^{-6}$, so climb remains significant after controlling for distance. The intercept differs from zero at the 5% level, but extrapolating to a zero-distance, zero-climb race has no useful physical interpretation.

The regression diagnostics show a very large positive residual for Knock Hill and a smaller positive outlier for Bens of Jura. The Q-Q plot has a pronounced upper tail, contradicting the assumed normal-error pattern. These observations can distort both the fitted coefficients and the estimated error variance. Check the source records, units, transcription and any unusual race conditions first, especially for Knock Hill; correct a demonstrated data error and refit. If the records are valid, consider omitted terrain effects, nonlinear distance/climb effects, a suitable response transformation or robust regression, and compare fits with and without influential observations. A diagnostic outlier alone is not a reason to silently delete a valid record.

Write $R_j=I-P_{-j}$. Minimizing the residual sum of squares first over the coefficients of $X_{-j}$ reduces the problem to minimizing $\Vert R_{j}Y-b{R}_j{X}_j{\Vert }^2$. Full column rank ensures $R_j{X}_j\ne 0$. Thus the partial regression formula and its variance areThis uses $R_j^T=R_j=R_j^2$ and the isotropic error covariance matrix ${\sigma }^{2}I$.

Since the span of $X_k$ is contained in the column space of $X_{-j}$, orthogonal projection onto the latter removes at least as much squared norm. Consequentlyand inversion proves the requested lower bound. Full rank makes the denominator positive.

Adding $c{1}_n$ to column $j\ge 2$ leaves the column space unchanged. Every old fitted vector is reproduced by keeping all slopes unchanged and replacing the regression intercept by ${\hat{\beta }}_1-c{\hat{\beta }}_j$. Uniqueness of the ordinary least squares estimators then proves that slope $j$ is unchanged. In particular all columns can be centered by such changes. Applying the previous projection argument to the centered design proves the bound with $X_j-{\bar{X}}_j{1}_n$ and $X_k-{\bar{X}}_k{1}_n$; the normalized inner product is now the usual sample correlation.

Each reported coefficient test is the two-sided Student's t-test of $H_0:{\beta }_j=0$ against $H_1:{\beta }_j\ne 0$, conditional on the other predictors. Under the null, ${\hat{\beta }}_j/(\hat{\sigma }\sqrt{(X^{T}X{)}_{jj}^{-1}})$ has a Student's t-distribution with $100-3=97$ degrees of freedom. At five percent the intercept is significant, but neither individual test slope is. The final F-test compares the full model against an intercept-only model: $H_0:{\beta }_{\mathrm{Maths}}={\beta }_{\mathrm{Stats}}=0$. It has $(2,97)$ degrees of freedom and rejects at five percent, since its $p$-value is $0.003166$. Thus the predictors jointly explain variation, without strong evidence separating their individual conditional effects.

The two test predictors have sample correlation $0.9371630$, although each has only moderate correlation with the final exam response. The variance inflation factor is $1/(1-0.9371630^2)\simeq 8.22$. Their nearly shared direction produces substantial multicollinearity, inflating the individual slope standard errors. The joint F-test can detect this common predictive direction even though either coefficient, given the other, has a large $p$-value.

Use the usual normal linear model $Y=X\beta +\epsilon$, with fixed full-column-rank design $X\in {\mathbb{R}}^{n\times p}$, $n>p$, and $\epsilon \sim N(0,{\sigma }^2{I}_n)$. These Gaussian distribution and constant-variance hypotheses are necessary for an exact finite-sample Student's t-test. Put $V=(X^{T}X{)}^{-1}$ and $\nu =n-p$. The vector of ordinary least squares estimators satisfiesWith $s^2=\Vert Y-X\hat{\beta }{\Vert }^2/\nu$, the test statistic isUnder the null hypothesis, $T$ has Student's t-distribution with $\nu$ degrees of freedom. A two-sided level-$5\%$ Student's t-test rejects when $|T|>t_{\nu ,0.975}$. If the intended test is one-sided against ${\beta }_0>0$, it rejects when $T>t_{\nu ,0.95}$; the statistical power conclusion below holds for either convention.

Assume $X$ has full column rank. The log-likelihood, up to constants, isThe ordinary least squares normal equations giveand maximizing over the variance gives the maximum-likelihood estimatorBecause $I-H$ is an orthogonal projection of rank $n-p$,Writing ${\chi }_{\nu ,q}^2$ for the $q$-quantile, a confidence interval of level $1-\alpha$ is

A regression tree chooses a threshold $s$ and constant predictions $c_L,c_R$ to minimize the squared-error criterionwhere $s$ ranges over thresholds producing two nonempty groups. For a fixed split, the least-squares minimizers are the two sample means,Thus the first split is the candidate threshold with the smallest sum of the two within-node residual sums of squares.

WriteThe ordinary least squares estimators givewhereThe denominator is positive because the sorted inputs are distinct and $m\ge 2$.

LetDifferentiating the Gaussian log-likelihood, equivalently minimizing the residual sum of squares, gives the ordinary least squares estimatorsassuming $S_{xx}>0$.