Apply the squared scalar triple product from cyclic cross products proved in c:Hence the reciprocal basis has scalar triple product $\boxed{{\mathbf{a}}^{\prime }\cdot ({\mathbf{b}}^{\prime }\times {\mathbf{c}}^{\prime })=1/V}$, with its sign retained.

The cross product $\mathbf{b}\times \mathbf{c}$ is perpendicular to both factors. ThusCyclic permutations give all the off-diagonal dot products between the original basis and its reciprocal basis.

Let $V=\mathbf{a}\cdot (\mathbf{b}\times \mathbf{c})\ne 0$. The scalar triple product is invariant under a cyclic permutation, so the definitions of the reciprocal basis give

The new score columns are orthogonal, since $Z^{T}Z=V^T{X}^{T}XV$ is diagonal. This decouples the slope normal equations; with positive eigenvalues and the stated error covariance, the two estimated score coefficients are uncorrelated. It also makes variance explained and possible dimension reduction explicit.

The trade-off is interpretation: each principal component is a mixture of the original variables, so its regression coefficient is less directly connected to changing one original input. Keeping both components is an invertible reparameterization and gives exactly the same fitted values and residual sum of squares. It does not repair a zero eigenvalue or remove instability associated with a very small one. Dropping a low-variance component may improve numerical stability and reduce estimator variance, but changes the model and can discard predictive information, since principal component analysis uses predictor variance rather than the response. This is the distinction between a full rotation and principal component regression with truncation.

Here both regressor columns are already centered, so $Z=XV$ and $X=Z{V}^T$. Writing $\beta =({\beta }_1,{\beta }_2{)}^T$, the linear predictor is $X\beta =Z{V}^T\beta$. ThusThe intercept stays ${\beta }_0$, because centering made the transformation a rotation of the regressor columns without an added translation.

The principal component scores are the projections of centered observations onto the loading directions:If the symbol $X$ already denotes centered data, this simplifies to $Z=XV$. The principal component loadings are directions; the principal component scores are coordinates of individual observations in those directions.

Let $X_c$ be the column-centered data matrix. Its sample covariance matrix is $S=X_c^T{X}_c/(n-1)$. The loadings matrix $V=(v_1,\dots ,v_p)$ has orthonormal eigenvectors of $S$ as columns, ordered by nonincreasing eigenvalues:Each principal component loading specifies the coefficients of a principal direction in the original variables. Signs are arbitrary, and equal eigenvalues permit orthogonal changes of basis within their eigenspaces.

Principal component analysis finds orthogonal directions of greatest sample variation, ordered by the variance explained. It can summarize correlated variables, reveal lower-dimensional structure, and provide a reduced representation by retaining only the first few principal components. Centering is standard; scaling may also be appropriate when the variables have incomparable units.

If the two regressors are strongly associated, the data may provide little support for varying one while holding the other fixed. Multicollinearity makes the separate slope estimates unstable, and exact linear dependence makes them unidentifiable. A conditional slope may also be an extrapolation beyond combinations actually observed.

Furthermore, a slope in a linear regression model is not automatically a causal effect. A causal interpretation needs appropriate design or assumptions excluding confounding; independence of the errors across observations and zero mean alone do not establish those assumptions. The coefficients describe the specified linear mean relation, but changing a regressor by intervention can have a different interpretation.

The centered regressors make the first normal equation read $n{\hat{\beta }}_0={\sum }_i{Y}_i$. DefineThe remaining equations are $A{\hat{\beta }}_1+B{\hat{\beta }}_2=u$ and $B{\hat{\beta }}_1+C{\hat{\beta }}_2=v$. Provided $D=AC-B^2>0$, matrix inversion givesBy the Cauchy-Schwarz inequality, $D\ge 0$. If $D=0$, the two regressor columns are linearly dependent, and their separate slopes are not uniquely identifiable. One still has well-defined linear least squares fitted values, but cannot use the displayed divisions by $D$.

Minimize the residual sum of squaresSetting its three partial derivatives to zero gives the normal equationsThese equations characterize the linear least squares minimizers, since $Q$ is a convex quadratic. A unique coefficient vector additionally requires full column rank of the design matrix.

Since ${\sum }_{i=1}^n{D}_i=0$, we have $D_1=-{\sum }_{i=2}^n{D}_i$. Squaring and adding the other residual squares givesTherefore $S^2$ is a measurable function of $(D_2,\dots ,D_n)$. By the block independence proved in part i, it is independent of $\overline{X}$. This is the independence of the normal sample mean and sample variance; pairwise independence alone would not justify passing to an arbitrary function of several residuals.

Each residual has mean zero and variance ${\sigma }^2(1-1/n)$. By linearity of expectation, without needing independence of residuals,Hence $S^2$ is an unbiased estimator of ${\sigma }^2$.

Put $D_j=X_j-\overline{X}$. The supplied linear-transformation result shows that $(\overline{X},D_2,\dots ,D_n)$ is jointly multivariate normal. Its marginal distributions areIndeed, $\mathrm{Cov}(X_j,\overline{X})={\sigma }^2/n$, soFor a jointly Gaussian vector, zero cross-covariance between blocks implies their independence. Thus $\overline{X}$ is independent of the entire vector $(D_2,\dots ,D_n)$, not merely of each coordinate separately. The residual coordinates themselves need not be independent: for $j\ne k$, their covariance is $-{\sigma }^2/n$.

The sample mean is exactly $N(\mu ,{\sigma }^2/n)$, so $(\overline{X}-\mu )/(\sigma /\sqrt{n})$ has the standard normal distribution. The given quantile gives probability $0.95$ to the interval $[-1.96,1.96]$. Inverting this pivot yields the exact $95\%$ confidence interval

For known ${\sigma }^2>0$, maximizing the normal likelihood over $\mu$ is equivalent to minimizing ${\sum }_i(X_i-\mu {)}^2$. The identityshows that the unique minimum occurs at $\boxed{\hat{\mu }=\overline{X}}$. Equivalently, the log-likelihood has second derivative $-n/{\sigma }^2<0$.

The central limit theorem givesWrite $z=z_{1-\alpha /2}$ for the indicated standard normal quantile. Inverting the approximate pivot inequality and substituting $Y/n=1/\hat{\theta }$ gives the approximate confidence intervalIts coverage is approximately $1-\alpha$ for large $n$. The intersection respects the positive rate parameter; asymptotically the lower endpoint is positive. This is an asymptotic interval, not an exact finite-sample confidence interval.

The inverse moments in part i giveThus the bias is positive, and direct expansion of the mean squared error yieldsFor comparison, the variance is $n^2{\theta }^2/[(n-1{)}^2(n-2)]$; adding the squared bias gives the same answer by the bias-variance decomposition.

Use the gamma density and the normalization integral for shape $n-1$:The same argument, valid for $n\ge 3$, gives $\mathbb{E}[Y^{-2}]={\theta }^2/[(n-1)(n-2)]$.

For the independent random sample, the likelihood function on its support is $L(\theta )={\theta }^n{e}^{-\theta Y}$, where $Y=\sum X_i$. Its log-likelihood is $\ell (\theta )=n\log \theta -\theta Y$, withAlmost surely $Y>0$, and the unique stationary point is consequently the unique global maximum:The exceptional all-zero sample has probability zero; if $Y=0$ were nevertheless supplied, there would be no finite maximizing rate.