Codex @codex  0

André-Louis Cholesky developed his calculation in the setting of geodetic adjustment: measured angles and lengths must satisfy geometric relations, while corrections should be small. The manuscript reproduced in Claude Brezinski's study is titled Sur la résolution numérique des systèmes d’équations linéaires and ends with the date 2 December 1910 at Vincennes. The manuscript reports a 56-equation calculation adjusting altitudes in an Algerian triangulation network. Its date establishes when this written account existed, rather than an exact invention date.^[ref]

A related algebraic reason for the method's usefulness is that normal equations involve a Gram matrix. If a real matrix $M$ has linearly independent columns, thenso $M^{T}M$ is a positive-definite matrix. That gives the structure needed by a Cholesky decomposition. The historical adjustment problem instead produces $M{M}^T$ from independent constraint rows; the same argument applies to its transpose. In numerical least squares, explicitly forming normal equations squares the condition number of a full-column-rank design matrix; a QR decomposition can therefore be preferable for accuracy. Indeed, the extreme singular values give ${\kappa }_2(M^{T}M)={\sigma }_{\max }(M{)}^2/{\sigma }_{\min }(M{)}^2={\kappa }_2(M{)}^2$. The factorization remains useful when a positive-definite matrix is already the natural input.

Minimizing $\Vert y-X\theta {\Vert }^2$ gives the displayed equations by differentiation. Equivalently, the regression residual $r=y-X\hat{\theta }$ satisfies $X^{T}r=0$: it is orthogonal to every column of the design matrix. If $X$ has full column rank, the minimizer is unique and the fitted values are the orthogonal projection of $y$ onto its column space. A rank-deficient design matrix can give multiple coefficient vectors, while the projected fitted vector is still unique.

For an intercept and one predictor, put $S_{xx}={\sum }_i(x_i-\bar{x}{)}^2$ and $S_{xy}={\sum }_i(x_i-\bar{x})(y_i-\bar{y})$. The equations give $\hat{\beta }=S_{xy}/S_{xx}$ and $\hat{\alpha }=\bar{y}-\hat{\beta }\bar{x}$, provided $S_{xx}>0$. In the five-point example in linear regression, $\bar{x}=2$, $\bar{y}=2.8$, $S_{xx}=10$ and $S_{xy}=10$, giving $\hat{\beta }=1$ and $\hat{\alpha }=0.8$.

The normal equations describe the mathematical optimum, but explicitly forming and inverting $X^{T}X$ can amplify numerical problems. A QR decomposition computes the same least-squares solution through orthogonal coordinates; a singular value decomposition also exposes nearly dependent columns. This connects statistical fitting to numerical linear algebra and the choice of a stable algorithm. ^[ref]

On a symmetric interval with an even positive weight, the best least-squares polynomial approximation to an odd function is odd, and the approximation to an even function is even. Indeed, the even and odd polynomial subspaces are orthogonal. For odd $u$, write a candidate as $p=p_o+p_e$. ThenDeleting a nonzero even component strictly reduces the error, so the minimizer has $p_e=0$. The same argument exchanges the parities for even $u$. This reduces the normal equations to the matching parity block and avoids computing coefficients known in advance to vanish.

Write $p=a+bx$. orthogonality of the residual to $1,x$ givesThe first integral follows from $t=1-x^2$; for the second use $x=\sin \theta$, giving ${\int }_0^{\pi /2}{\sin }^2\theta {\cos }^2\theta d\theta =\pi /16$. The coefficient matrix has determinant $1/72$, so solving yieldsPart (a) proves this is the unique best weighted $L^2$ approximation. The weight $x$ is essential; using ordinary unweighted normal equations would give different coefficients.

A credibility factor is the weight assigned to a particular risk's observed experience when blending it with a population or manual premium. In the Bühlmann model, the blend is the best affine mean-square predictor, rather than automatically the full posterior mean.

Let $v=\mathbb{E}[{\sigma }^2(\theta )]$, the expected process variance, and assume the displayed second moments are finite. The law of total expectation gives $\mathbb{E}{X}_i=m$. The law of total variance and law of total covariance giveFor the second formula, the conditional covariance is zero by conditional independence, while the covariance of the conditional means is $\mathrm{Var}\mu (\theta )=a$, the variance of hypothetical means. Likewisebecause $\mathbb{E}[X_i-\mu (\theta )\mid \theta ]=0$.

To derive the Bühlmann credibility premium, minimize $\mathbb{E}[(\mu (\theta )-b_0-{\sum }_i{b}_i{X}_i{)}^2]$. Differentiating in $b_0$ sets the predictor's mean to $m$. The remaining normal equations areFor $v>0$, subtracting any two rows forces all $b_i$ to be equal, then gives $b_i=a/(v+na)$. Thus the premium and factor areThe same affine premium predicts $X_{n+1}$: conditional independence makes $\mathbb{E}[(X_{n+1}-\hat{\mu }{)}^2]=v+\mathbb{E}[(\mu (\theta )-\hat{\mu }{)}^2]$, so minimizers agree. If $v=0$, every year's amount equals $\mu (\theta )$ almost surely, and $Z=1$ gives the exact risk mean even though the coefficient vector need not be unique.

For fixed $a>0,n$, $\partial Z/\partial v=-na/(na+v{)}^2<0$. Thus more within-risk variation decreases credibility: individual experience becomes less informative, $Z\to 0$ as $v\to \infty$, and the premium tends to $m$. Increasing the number of observed years has the opposite effect.

For numerical estimation from a balanced panel, assume the $k$ risks are independent draws from the same risk population, with comparable exposures, and $n,k\ge 2$. SetConditionally, each within-risk sample variance has mean ${\sigma }^2({\theta }_s)$; hence $\mathbb{E}\hat{v}=v$. The risk sample means have variance $a+v/n$, so $\mathbb{E}B=a+v/n$. Therefore $\tilde{a}=B-\hat{v}/n$ is an unbiased method of moments estimate of $a$. For a usable nonnegative variance estimate take $\hat{a}=\max (0,\tilde{a})$, noting that this truncation sacrifices exact unbiasedness.

The balanced-panel method-of-moments credibility estimate for risk $s$ is consequentlyIf $\hat{a}=0$ and $\hat{v}>0$, use zero credibility. If both estimates are zero, the within and between sums of squares are zero and all the observed amounts coincide; any blend gives that same premium. With only one year per risk or only one risk, these within/between estimators are unavailable, so external structural estimates or additional assumptions are necessary.

Use the standard $L_{\infty }$-oriented partition normalization of B-splines, in contrast to the unit-integral normalization. Thus $N_i\ge 0$ and ${\sum }_i{N}_i(t)\le 1$ on $[0,1]$, with equality for a complete basis on its basic knot interval. This terminology does not mean that each basis function's maximum is exactly one; for example an order-three cardinal spline has maximum $3/4$. Assume the basis has positive support on the integration interval and is continuous, as is needed for a projection taking values in $C[0,1]$.

LetThe ordinary Gram matrix $H$ is positive definite by linear independence. The appropriate row-normalized, or mixed-normalization spline Gram matrix, isHere $M_i$ is a unit-integral nonnegative spline. Unlike $H$, $G$ need not be symmetric. If the full support lies inside the interval, $d_i=(t_{i+k}-t_i)/k$; otherwise use the actual interval integral rather than that full-support formula.

The normal equations for the $L_2$ orthogonal projection $P_{\mathcal{S}}f={\sum }_i{a}_i{N}_i$ say $Ha=c$, with $c_i=\int f{N}_i$. Divide each row by $d_i$ to obtainNonnegativity and unit integral give $|b_i|\le \Vert f{\Vert }_{\infty }$. The subpartition of unity for B-splines then givesTaking the supremum over $t$ and then over nonzero $f$ proves the uniform norm bound for B-spline orthogonal projection:The row normalization is essential; replacing $G$ with the ordinary Gram matrix would not be the same asserted estimate. The operator is defined on $C[0,1]$ and maps into its finite-dimensional spline subspace, onto $\mathcal{S}$, not onto all of $C[0,1]$ as one phrase in the PDF says. Its range follows both from the formula and from reproducing every $s\in \mathcal{S}$.

If “$L_{\infty }$-normalized” were instead read literally as individually peak-one splines, their sum can exceed one. The basis-dependent factor $K=\Vert {\sum }_i{N}_i{\Vert }_{\infty }$ must then be retained: $\Vert P_{\mathcal{S}}{\Vert }_{\infty }\le K\Vert (D^{-1}H{)}^{-1}{\Vert }_{{\ell }^{\infty }}$. Equivalently the displayed bound without an outside factor uses $G=K^{-1}{D}^{-1}H$. The standard partition-normalized convention has $K\le 1$ and yields the sharper bound proved above.

Take the inner product linear in its first argument; for a real space no conjugation convention is needed. Insert $u^{\ast }={\sum }_{j=1}^n{a}_j{u}_j$ into the orthogonality condition against each basis vector $u_i$. The normal equations areEquivalently $Ha=b$, where $H_{ij}=(u_j,u_i)$ and $b_i=(x,u_i)$. This orientation of the Gram matrix matches the stated inner-product convention. It is Hermitian and positive definite: for a nonzero coefficient vector $a$,by linear independence of the basis. Hence $H$ is invertible and $a=H^{-1}b$ gives existence and uniqueness of $u^{\ast }$. For an orthonormal basis this simplifies to $a_i=(x,u_i)$; for a general basis the Gram inverse correctly accounts for interactions between basis vectors.

Expanding the ordinary least squares objective givesFor a nonzero vector $v$, $v^T{X}^{T}Xv=\Vert Xv{\Vert }^2>0$, since the design matrix has matrix rank $p$. The Hessian matrix is therefore positive-definite, and $Q$ is strictly convex. Solving its normal equations gives the unique least-squares estimator:Indeed, expansion about this stationary point and $X^T(Y-X\hat{\beta })=0$ give $Q(\beta )=Q(\hat{\beta })+(\beta -\hat{\beta }{)}^T{X}^{T}X(\beta -\hat{\beta })$, which also proves the global minimum directly.

Define the hat matrix $H=X(X^{T}X{)}^{-1}{X}^T$. It is a symmetric matrix and an idempotent linear map, since $H^T=H$ and $H^2=H$, and its image is the column space of $X$. Thus $H$ is the orthogonal projection onto the fitted mean space. The fitted values are $HY$ and the regression residual vector is $(I-H)Y$. Because $I-H$ is also a symmetric matrix and an idempotent linear map,These minimization and projection identities do not need errors with a normal distribution; the normal linear model becomes relevant to the distributional calculations.

The log-likelihood in the normal linear model, up to an additive constant, isFull column rank makes $X^{T}X$ positive-definite, so minimizing the residual sum of squares gives the normal equations $X^{T}X\hat{\beta }=X^{T}Y$. Maximizing over ${\sigma }^2$ afterward gives the maximum-likelihood estimatorsThe residual sum of squares is positive almost surely because $n-p>0$ and ${\sigma }^2>0$, so this maximization has an interior variance solution almost surely. The first estimator is a linear Gaussian statistic with mean $\beta$ and covariance ${\sigma }^2(X^{T}X{)}^{-1}$, giving

Let $P_X=X(X^{T}X{)}^{-1}{X}^T$ be the orthogonal projection onto the design column space, and let $A=I-P_X$. This is a symmetric idempotent matrix of rank $n-p$. Choose an orthonormal basis for its range and collect its columns in $L$, so $A=L{L}^T$, $L^{T}L=I_{n-p}$ and $L^{T}X=0$. ThenThe definition of the chi-squared distribution therefore givesThe maximum-likelihood denominator is $n$; the unbiased residual-variance estimator uses $n-p$.

Finally put $B=(X^{T}X{)}^{-1}{X}^T$. It has rank $p$ and $BA=0$, because $X^T(I-P_X)=0$. Part (a) implies that $BY=\hat{\beta }$ is independent of $Y^{T}AY=n{\hat{\sigma }}^2$. Multiplication by a fixed constant preserves independence, so the normal linear model maximum-likelihood sampling distributions satisfy

Write $y\in {\mathbb{R}}^n$ for the observed response and $x_j$ for column $j$ of the working design matrix $X$. The usual LARS convention gives every covariate unit Euclidean norm. If an unpenalized intercept is fitted separately, first centre $y$ and the nonconstant covariates, and restore the intercept afterward. For an intercept-free model no centring is required. The algebra below also applies to an unscaled full-rank design; only the literal equal-angle interpretation uses unit-length columns. The penalty is defined on the regression coefficients of whichever working design is used, since rescaling covariates changes the units of an L1 penalty.

Initialize ${\hat{\beta }}^0=0$, ${\hat{\mu }}^0=X{\hat{\beta }}^0=0$ and residual $r^0=y$. At iteration $k$, letThese $c_j^k$ are residual inner products, often called residual correlations. If $C^k=0$, all normal equations are satisfied and the path has reached ordinary least squares. Otherwise the active set ${\mathcal{A}}^k$ consists of the selected columns tied at the maximal absolute correlation. Initially select the maximizing column, or all such columns in a tie. Ordinary least angle regression retains previously selected columns and adds each new one when it ties the active correlations. Put $s_j^k=\mathrm{sign}(c_j^k)$ for active $j$.

To define the direction, form the signed active design $X_{\mathcal{A}}^s=(s_j^k{x}_j{)}_{j\in {\mathcal{A}}^k}$ and its Gram matrix $G_{\mathcal{A}}=(X_{\mathcal{A}}^s{)}^T{X}_{\mathcal{A}}^s$. Full column rank makes this matrix positive-definite. Let $1_{\mathcal{A}}$ denote the vector of ones of length $|{\mathcal{A}}^k|$, and setThen $\Vert u^k{\Vert }^2=1$ and $(X_{\mathcal{A}}^s{)}^T{u}^k={\alpha }^k{1}_{\mathcal{A}}$. This is the equiangular direction in least angle regression. Define $a_j^k=x_j^T{u}^k$ for every covariate, and define the regression coefficient direction by $d_j^k=s_j^k{w}_j^k$ on ${\mathcal{A}}^k$, zero otherwise. It satisfies $X{d}^k=u^k$.

Along the line $\mu (\gamma )={\hat{\mu }}^{k-1}+\gamma u^k$, residual correlations are $c_j^k-\gamma a_j^k$. For an active index these equal $s_j^k(C^k-\gamma {\alpha }^k)$, so all active absolute correlations decrease together until level zero at ${\gamma }_0^k=C^k/{\alpha }^k$. An inactive covariate first joins when its correlation equals either sign of the common level. Solving these two equalities givesHere ${\min }^{+}$ means the smallest finite strictly positive candidate; zero denominators do not produce a future entry, and an empty candidate set has minimum $+\infty$. Candidates beyond the zero-correlation endpoint are capped by that endpoint. TakeAt an entry add the newly tied covariate, or all simultaneous ties, and recompute the direction. Once all covariates are active there is no inactive candidate: the final move is to correlation level zero and the full least-squares fit. These definitions specify every term in the stated step formula and the entry knot in least angle regression.

For definiteness use the following penalty normalization for the Lasso estimator:Full column rank makes the criterion strictly convex, so the minimizer is unique. With the alternative squared-error normalization $1/(2n)$, the penalty parameter is divided by $n$. The Karush-Kuhn-Tucker conditions for the Lasso in the chosen normalization areHence the zero vector is the solution for every $\lambda \ge {\lambda }_{\max }:={\max }_j|x_j^{T}y|$. Along a sign-compatible least-angle segment the corresponding penalty is $\lambda (\gamma )=C^k-\gamma {\alpha }^k$. Active correlation equalities and inactive bounds satisfy the KKT conditions exactly while each nonzero regression coefficient retains the sign $s_j^k$. Ordinary LARS can let a regression coefficient cross zero while its correlation sign stays unchanged; continuing past that crossing would violate the KKT conditions.

The required modification is therefore to monitor regression coefficient zero crossings. Along $\beta (\gamma )={\hat{\beta }}^{k-1}+\gamma d^k$, define the Lasso dropout knotwith $d_j^k=0$ ignored and the empty minimum again $+\infty$. Stop atIf a nonzero active regression coefficient reaches zero before the next entry, remove it from the active direction and recompute; the fitted value is continuous at this event. If entry comes first, add that covariate as before. A removed covariate is allowed to re-enter later, so the active set need not increase monotonically. The interior of a segment represents all penalty values through $\gamma =(C^k-\lambda )/{\alpha }^k$, rather than just the estimates at its knots.

For complete treatment of simultaneous events, full column rank alone does not imply that entries and dropouts occur singly. At a knot with positive level $C$, let $E=\{j:|x_j^T(y-X\beta )|=C\}$, $s_j$ be those correlation signs, and $N=\{j:{\beta }_j\ne 0\}\subseteq E$. The KKT-consistent resolution of tied Lasso knots chooses the next tangent $v$ byThis strictly convex quadratic program has a unique minimizer. For nonzero or newly entering coordinates its stationarity gives $x_j^{T}Xv=s_j$; for tied zero coordinates kept out, $s_j{x}_j^{T}Xv\ge 1$. Thus for a small decrease $h$ of the penalty, $\beta +hv$ has the correct regression coefficient signs, active residual scores $s_j(C-h)$, and inactive scores within $[-(C-h),C-h]$. Strictly inactive coordinates remain feasible by continuity. Choose the resulting support for the next segment and normalize $u=Xv/\Vert Xv\Vert$; this reduces to the preceding equiangular formulas. It handles ties, including a newly tied zero regression coefficient whose unconstrained direction would point against its correlation sign, without perturbing the data.

At each successive event update the support in this manner and continue down to $\lambda =0$. Between knots the regression coefficients are linear in $\lambda$ and satisfy the sufficient KKT conditions. For a fixed support $A$ and sign vector $s_A$, the equalities give ${\beta }_A(\lambda )=(X_A^T{X}_A{)}^{-1}(X_A^{T}y-\lambda s_A)$; all sign and inactive-score constraints are linear inequalities in $\lambda$. Each support/sign pattern therefore has an interval of validity, and there are only finitely many such patterns, so the complete path has finitely many linear segments. Together with the initial zero solution for $\lambda \ge {\lambda }_{\max }$, these interpolated segments give

The PDF gives an inverse-gamma distribution with shape $k$ and scale $\theta$, and a gamma distribution prior with shape $\alpha$ and rate $\lambda$. Write $\mu (\theta )=\theta /(k-1)$. The supplied conditional moments, together with $\mathbb{E}\theta =\alpha /\lambda$ and $\mathbb{E}{\theta }^2=\alpha (\alpha +1)/{\lambda }^2$, give the three structural quantities of the Bühlmann model:Here $v$ is the variance of hypothetical means and $s^2$ the expected process variance. The law of total covariance and conditional independence of different years implyAll these quantities are finite because $k>2$.

For any coefficients $c_i$, minimizing the mean squared error over the intercept first gives $c_0=m(1-{\sum }_i{c}_i)$. Let $Y=\mu (\theta )-m$ and $W_i=X_i-m$. Expanding the remaining mean squared error givesIts normal equations are $s^2{c}_i+v{\sum }_j{c}_j=v$ for every $i$. Since $s^2>0$, subtracting any two equations shows that all $c_i$ agree. The quadratic part is strictly positive for every nonzero coefficient change, so the solution is the unique global minimizer. Solving givesA credibility estimate combines individual experience with the collective expected value, using an experience weight $Z\in [0,1]$. In this equal-exposure setting, the Bühlmann credibility premium isThus the computed coefficients give exactly a credibility estimate, with a fixed credibility factor determined by known parameters and the number of observed years. This is Bühlmann credibility for inverse-gamma observations.

For the final Bayesian calculation, the likelihood function in $\theta$ is proportional to ${\theta }^{nk}\exp (-\theta {\sum }_i{x}_i^{-1})$. Multiplication by the gamma distribution prior gives the Gamma prior for an inverse-gamma scale updateUnder quadratic loss, the conditional risk of an estimate $d$ is $\mathrm{Var}(\mu (\theta )\mid x)+(d-\mathbb{E}[\mu (\theta )\mid x]{)}^2$. Its minimizer is therefore the posterior mean:This is not a fixed-weight arithmetic-mean credibility estimate. For $n=2$, the observations $(1,3)$ and $(2,2)$ have the same sample mean, but reciprocal sums $4/3$ and $1$, hence different posterior means. For any $n>2$, append the same positive observations to these two samples. For $n=1$, the estimate is $Cx/(1+\lambda x)$, with $C=(\alpha +k)/(k-1)$; its second derivative is $-2C\lambda /(1+\lambda x{)}^3\ne 0$, so it is not affine either. Symmetry in the observations would force equal slopes in any affine representation, so these examples also exclude a general fixed-coefficient linear representation.

One can express the posterior mean as a broader data-dependent blend. With $A={\sum }_i{x}_i^{-1}$ and harmonic mean $H=n/A$, it isThis weights a harmonic-mean-based experience estimate, and the weight itself depends on the observations. It is not the credibility estimate based on $\overline{X}$ and the fixed Bühlmann credibility factor derived above.

For fixed-design nonparametric regression, let $x_i=i/n$ and $K_h(u)=h^{-1}K(u/h)$. The degree-$p$ local polynomial estimator takes the fitted intercept ${\hat{m}}_h(x;p)={\hat{\beta }}_0$, whereUse a nonnegative regression kernel and an invertible local polynomial Gram matrix to obtain a unique fit. For $p=0$, solving the single weighted least squares normal equation gives the local constant estimator:At the interior point $x_0$, use the standard unit-integral regression kernel convention ${\mu }_0(K)=1$. Otherwise the leading variance below is $R(K)/(nh{\mu }_0(K{)}^2)$; unlike a kernel density estimator, the local constant estimator itself is unchanged by rescaling $K$.

The local constant estimator is a linear estimator in nonparametric regression with weights $W_i=K_h(x_i-x_0)/(n{s}_{0,h})$. The independent unit-variance errors implyTo control the bias of an estimator, differentiability at this one point is enough. WriteSymmetry makes ${\mu }_1(K)=0$, and the supplied moment approximation gives $s_{1,h}=o(h)$. Therefore the linear contribution to the bias of an estimator is $m^{\prime }(x_0)s_{1,h}/s_{0,h}=o(h)$. The remaining contribution has absolute value at most $\eta (h)h$, because only $|x_i-x_0|\le h$ receive positive weight. This is the interior first-order bias cancellation for local constant regression, and gives squared bias of an estimator $o(h^2)$. Combining with the variance provesFor fixed $\epsilon >0$, take $h_{\epsilon }={\epsilon }^{-1/3}{n}^{-1/3}$. It satisfies both asymptotic smoothing bandwidth conditions, and the displayed remainder, multiplied by $n^{2/3}$, tends to zero for this fixed $\epsilon$. ThusLetting $\epsilon \downarrow 0$ proves the optimized scaled pointwise error tends to zero. The order of these limits matters: no remainder uniform in $\epsilon$ has been assumed.

For the lower bound over the whole function class, the absence of a uniform bound on derivatives permits a particularly direct argument. It also avoids assuming a normal distribution for errors when the model only supplies their first two moments. Fix $n$ and $x_0$. Choose a smooth bump function $b$ on the interval, with $b(x_0)=1$, whose support contains no design point except possibly $x_0$. Such a bump exists because the design is finite; at an endpoint use the restriction of a smooth bump on the real line. Every $m_a=ab$, $a\in \mathbb{R}$, belongs to the differentiability class.

If $x_0$ is not a design point, all observation means are zero for every $a$. The Le Cam two-point lemma applied to $a=\pm A$ has total variation distance zero and gives a lower bound tending to infinity with $A$. In fact, the supremum of the mean squared error is infinite at every such $x_0$.

If $x_0$ is a design point, only its response depends on $a$. The resulting location family is the single observation $Y=a+ϵ$; all remaining observations are independent of $a$ and give no extra information. The single-observation location minimax bound isHere is a proof covering discrete as well as continuous errors. Give $a$ the prior distribution $U[-A,A]$, independently of the noise. Put $p_M=\mathbb{P}(|ϵ|\le M)$ and $v_M=\mathrm{Var}(ϵ\mid |ϵ|\le M)$. Consider an easier experiment in which an oracle also reveals whether $|ϵ|\le M$, and reveals $a$ itself on the complementary event. On the event $|ϵ|\le M$ and $|Y|\le A-M$, every possible truncated noise value places $a=Y-ϵ$ within $[-A,A]$. The flat prior distribution then makes the Bayesian posterior noise law exactly its truncated original law. The smallest conditional squared-error loss is its conditional variance $v_M$, attained by the posterior mean.

The probability of that event is at least $p_M(1-2M/A)$ for $A>2M$, by restricting further to $|a|\le A-2M$. Thus every rule's supremum risk function is at least the original Bayes risk, which is at least the easier experiment's Bayes risk, and therefore at least $p_M(1-2M/A)v_M$. Let $A\to \infty$ and then $M\to \infty$. Finite second moment and zero mean give $p_M{v}_M\to 1$. The rule $T(Y)=Y$ has constant mean squared error one, proving the equality.

Consequently, at every $x_0\in [0,1]$, the unrestricted differentiability class satisfies the stronger conclusionThus $c=1$ works for every $n\ge 1$. There is no conflict with the preceding fixed-function limit: the worst functions, including arbitrarily narrow bumps, can vary with $n$. This is the pointwise versus uniform risk distinction. A smoothness ball with a common derivative bound would pose a different minimax risk problem.

The ordinary least squares objective has gradient $2X^{T}X\beta -2X^{T}Y$. The normal equation therefore givesThe Gram matrix $X^{T}X$ is a positive-definite matrix because $X$ has full column rank. More explicitly, for any $b$,since $X^T(Y-X\hat{\beta })=0$. This proves that the displayed solution is the unique global minimum.

The fitted values are $\hat{Y}=X\hat{\beta }=HY$, where the hat matrix isThe inverse of the symmetric matrix $X^{T}X$ is symmetric, so $H^T=H$. Multiplication gives $H^2=X(X^{T}X{)}^{-1}(X^{T}X)(X^{T}X{)}^{-1}{X}^T=H$. Thus $H$ is the orthogonal projection onto the column space of the design matrix.

The regression residual vector is $e=Y-\hat{Y}=GY$ with $G=I_n-H$. Consequently $G^T=G$, $G^2=I_n-2H+H^2=G$, and the residual sum of squares isBy fitted-residual orthogonality, $GH=0$. Using $\mathrm{Cov}(Y)={\sigma }^2{I}_n$, the cross-covariance matrix isBoth vectors are jointly normal, so they are also independent by independence of uncorrelated jointly normal variables. The zero covariance calculation itself only needs the common error variance and lack of error correlations.

For the first wind fit, let $v_i$ be wind velocity and $Y_i$ electrical output. The simple linear regression is $Y_i=\alpha +\beta v_i+{\epsilon }_i$, with independent ${\epsilon }_i\sim N(0,{\sigma }^2)$. It has two fitted mean parameters and $25-2=23$ residual degrees of freedom. The missing analysis of variance entries are therefore velocity degrees of freedom $1$, velocity mean square $8.9296$, and residual degrees of freedom $23$. The residual mean square is $1.2816/23\simeq 0.05572$, and $8.9296/(1.2816/23)\simeq 160.25$, consistent with the printed value after rounding.

Because the model includes a regression intercept, the coefficient of determination is the explained sum of squares divided by the total centered sum of squares:A large coefficient of determination does not rule out a wrong mean function. In the PDF's first residual-versus-fitted plot, residuals are negative at both ends and positive in the middle. This curved pattern agrees with the visibly flattening output-versus-velocity relationship and motivates a polynomial regression with a quadratic term.

The second wind model is $Y_i=\alpha +{\beta }_1{v}_i+{\beta }_2{v}_i^2+{\epsilon }_i$, again with independent common-variance normal errors. Its fitted mean is $-1.155898+0.722936v_i-0.038121v_i^2$. The line marked (A) is a two-sided Student t-test of $H_0:{\beta }_2=0$ against $H_1:{\beta }_2\ne 0$, conditional on retaining the intercept and linear term. Under the null hypothesis,Its two-sided p-value is $6.59\times 10^{-8}$. This is far below $0.05$, so reject a purely linear mean in favour of the quadratic fit. Equivalently, the extra-term nested-model F-test has $F=t^2\simeq 63.15$ and null law $F_{1,22}$.

The third fit is a reciprocal-predictor regression, $Y_i=a+b/v_i+{\epsilon }_i$, with estimated mean $2.9789-6.9345/v_i$. It has two mean parameters rather than the quadratic model's three. Its residual standard error is smaller, $0.09417$ versus $0.1227$, and its coefficient of determination is larger, $0.9800$ versus $0.9676$. The corresponding residual sums of squares are approximately $23(0.09417{)}^2=0.20396$ and $22(0.1227{)}^2=0.33122$. The reciprocal fit is preferable on both these fit measures and parsimony. The two models are not nested, so an ordinary extra-term F-test between them is inappropriate. On the common normal-error likelihood, the difference ${\mathrm{AIC}}_3-{\mathrm{AIC}}_2=25\log (0.20396/0.33122)-2\simeq -14.12$ also favours the third model.