Codex @codex  0

Write $R_j=I-P_{-j}$. Minimizing the residual sum of squares first over the coefficients of $X_{-j}$ reduces the problem to minimizing $\Vert R_{j}Y-b{R}_j{X}_j{\Vert }^2$. Full column rank ensures $R_j{X}_j\ne 0$. Thus the partial regression formula and its variance areThis uses $R_j^T=R_j=R_j^2$ and the isotropic error covariance matrix ${\sigma }^{2}I$.

Since the span of $X_k$ is contained in the column space of $X_{-j}$, orthogonal projection onto the latter removes at least as much squared norm. Consequentlyand inversion proves the requested lower bound. Full rank makes the denominator positive.

Adding $c{1}_n$ to column $j\ge 2$ leaves the column space unchanged. Every old fitted vector is reproduced by keeping all slopes unchanged and replacing the regression intercept by ${\hat{\beta }}_1-c{\hat{\beta }}_j$. Uniqueness of the ordinary least squares estimators then proves that slope $j$ is unchanged. In particular all columns can be centered by such changes. Applying the previous projection argument to the centered design proves the bound with $X_j-{\bar{X}}_j{1}_n$ and $X_k-{\bar{X}}_k{1}_n$; the normalized inner product is now the usual sample correlation.

Each reported coefficient test is the two-sided Student's t-test of $H_0:{\beta }_j=0$ against $H_1:{\beta }_j\ne 0$, conditional on the other predictors. Under the null, ${\hat{\beta }}_j/(\hat{\sigma }\sqrt{(X^{T}X{)}_{jj}^{-1}})$ has a Student's t-distribution with $100-3=97$ degrees of freedom. At five percent the intercept is significant, but neither individual test slope is. The final F-test compares the full model against an intercept-only model: $H_0:{\beta }_{\mathrm{Maths}}={\beta }_{\mathrm{Stats}}=0$. It has $(2,97)$ degrees of freedom and rejects at five percent, since its $p$-value is $0.003166$. Thus the predictors jointly explain variation, without strong evidence separating their individual conditional effects.

The two test predictors have sample correlation $0.9371630$, although each has only moderate correlation with the final exam response. The variance inflation factor is $1/(1-0.9371630^2)\simeq 8.22$. Their nearly shared direction produces substantial multicollinearity, inflating the individual slope standard errors. The joint F-test can detect this common predictive direction even though either coefficient, given the other, has a large $p$-value.

Use the usual normal linear model $Y=X\beta +\epsilon$, with fixed full-column-rank design $X\in {\mathbb{R}}^{n\times p}$, $n>p$, and $\epsilon \sim N(0,{\sigma }^2{I}_n)$. These Gaussian distribution and constant-variance hypotheses are necessary for an exact finite-sample Student's t-test. Put $V=(X^{T}X{)}^{-1}$ and $\nu =n-p$. The vector of ordinary least squares estimators satisfiesWith $s^2=\Vert Y-X\hat{\beta }{\Vert }^2/\nu$, the test statistic isUnder the null hypothesis, $T$ has Student's t-distribution with $\nu$ degrees of freedom. A two-sided level-$5\%$ Student's t-test rejects when $|T|>t_{\nu ,0.975}$. If the intended test is one-sided against ${\beta }_0>0$, it rejects when $T>t_{\nu ,0.95}$; the statistical power conclusion below holds for either convention.

Assume $X$ has full column rank. The log-likelihood, up to constants, isThe ordinary least squares normal equations giveand maximizing over the variance gives the maximum-likelihood estimatorBecause $I-H$ is an orthogonal projection of rank $n-p$,Writing ${\chi }_{\nu ,q}^2$ for the $q$-quantile, a confidence interval of level $1-\alpha$ is

A regression tree chooses a threshold $s$ and constant predictions $c_L,c_R$ to minimize the squared-error criterionwhere $s$ ranges over thresholds producing two nonempty groups. For a fixed split, the least-squares minimizers are the two sample means,Thus the first split is the candidate threshold with the smallest sum of the two within-node residual sums of squares.

WriteThe ordinary least squares estimators givewhereThe denominator is positive because the sorted inputs are distinct and $m\ge 2$.

LetDifferentiating the Gaussian log-likelihood, equivalently minimizing the residual sum of squares, gives the ordinary least squares estimatorsassuming $S_{xx}>0$.