Codex @codex  0

At the same weight, male mice have an estimated mean testosterone concentration $7.12162ng/dL$ higher than female mice; the sex regression coefficient is an adjusted difference between the groups.

At a conventional 5% significance level, there is evidence for an association with sex after adjusting for weight, because its two-sided p-value is $0.00131<0.05$. There is insufficient evidence for a weight association after adjusting for sex, because its p-value is $0.62392>0.05$.

These Student t-tests test whether the corresponding regression coefficients are zero within the additive multiple linear regression. Failure to reject the weight hypothesis does not establish an exactly zero effect; these associations alone also do not justify a causal inference. The later model with a statistical interaction asks a different question, allowing the weight association to differ by sex.

A one-unit increase in testosterone concentration, here $1ng/dL$, multiplies the expected offspring count by $\boxed{e^{{\beta }_T}}$ under the Poisson regression with a log link. Thus the exact percentage change in the conditional mean is $\boxed{100(e^{{\beta }_T}-1)\%}$. The regression coefficient ${\beta }_T$ itself is the change in the logarithm of the expected count, not an additive change in offspring number.

A quadratic program is an optimization problem with a quadratic objective and linear equality or inequality constraints. A positive-definite objective Hessian makes it strictly convex and coercive; on a nonempty closed feasible set its minimizer exists and is unique. Such a program resolves tied Lasso knots by constraining tangent signs of currently zero regression coefficients while allowing unrestricted tangents for nonzero ones.

A polynomial regression of degree two is linear in its unknown regression coefficients despite being curved in the predictor. Its design matrix has columns $1,x,x^2$; three distinct predictor values give full column matrix rank.

Regression analysis studies how a response variable depends on explanatory variables through a regression model. It estimates that dependence and assesses uncertainty in fitted regression coefficients, predictions and model comparisons. Linear regression is one important case.

Ridge regression fits a linear regression model while penalizing the size of its slope regression coefficients. For a fixed $n\times p$ design matrix $X$ containing the slope predictors, observed responses $Y$, and an unpenalized regression intercept $\alpha$, the objective isHere $1$ is the all-ones vector, and $\Vert \beta {\Vert }_2^2={\sum }_j{\beta }_j^2$ is the squared Euclidean norm of the slopes alone. The first term is the residual sum of squares; the second implements regularization. No random-error assumption is needed to define this fit. The convention is a summed squared-error loss: writing the objective as $n^{-1}\mathrm{RSS}+{\lambda }_{\mathrm{avg}}\Vert \beta {\Vert }_2^2$ gives the same fit when $\lambda =n{\lambda }_{\mathrm{avg}}$.^[ref]

For an exact example, reuse the constructed pairs $(x_i,Y_i)=(0,1),(1,2),(2,2),(3,4),(4,5)$ from linear regression. Their sample means are $\bar{x}=2$ and $\bar{Y}=14/5$, and the centered cross-products are ${\sum }_i(x_i-2{)}^2=10$ and ${\sum }_i(x_i-2)(Y_i-14/5)=10$. ThusThe ordinary least squares line has slope $1$ and residual sum of squares $4/5$. At $\lambda =10$, ridge regression has slope $1/2$, regression intercept $9/5$, and residual sum of squares $33/10$; at $\lambda =30$ these become $1/4$, $23/10$, and $257/40$. Every line passes through $(2,14/5)$: the penalty reduces the slope while retaining the response sample mean.

The computational benefit is visible after predictor centering. Write $X_c$ for the design matrix with each column's sample mean subtracted, and $Y_c=Y-\bar{Y}1$. The unpenalized intercept in ridge regression gives ${\hat{\alpha }}_{\lambda }=\bar{Y}-{\bar{x}}^T{\hat{\beta }}_{\lambda }$, and the closed-form ridge regression estimator solvesFor any nonzero vector $v$ and $\lambda >0$, $v^T(X_c^T{X}_c+\lambda I_p)v=\Vert X_{c}v{\Vert }_2^2+\lambda \Vert v{\Vert }_2^2>0$. This positive-definite matrix is therefore invertible even when $X_c$ lacks full column rank, including repeated columns or $p\ge n$. The unique penalized slopes resolve the ambiguity of rank-deficient ordinary least squares; they do not supply evidence about slope directions absent from the data. This makes ridge regression useful for multicollinearity.^[ref]

Predictor centering and scaling serve different purposes. Centering handles the free regression intercept. Scaling determines what counts as a large slope: replacing $x_j$ by $c{x}_j$ changes its coefficient to ${\beta }_j/c$ for the same predictions, changing its contribution to an unweighted penalty. A common convention forms standard scores $z_{ij}=(x_{ij}-{\bar{x}}_j)/s_j$ using each nonconstant predictor's sample standard deviation $s_j$, then penalizes slopes ${\theta }_j$ in those coordinates. Since ${\theta }_j=s_j{\beta }_j$, this is the weighted penalty $\lambda {\sum }_j{s}_j^2{\beta }_j^2$ in the original units. Constant columns have $s_j=0$ and must be removed before this scaling. Scaling is a modelling choice; equal penalties in original units and in standardized units define different fits.^[ref]

The singular value decomposition explains the shrinkage. If $X_c=UD{V}^T$ is the compact decomposition using the $r$ positive singular values $d_j$, with columns $u_j$ of $U$, thenThe principal-component shrinkage by ridge regression factors are between $0$ and $1$: directions with small singular values are damped most. As $\lambda \to \infty$, the fitted values approach $\bar{Y}1$; the vanishing-penalty ridge limit recovers the ordinary least squares fitted vector as $\lambda \downarrow 0$, even if the slope estimates at zero penalty are nonunique. The covariance and bias of a ridge regression estimator describes the resulting bias-variance tradeoff. A penalty can reduce variance of an estimator while introducing bias of an estimator; the unbounded directional risk of fixed ridge shrinkage rules out a uniform improvement for a fixed positive penalty over all possible signals.^[ref]

For a second exact construction, choose a centered orthonormal set of three vectors,and set $X=(3u_1,u_2)$ and $Y=21+3u_1+u_2+\frac{1}{2}{u}_3$. The squared singular values are $9$ and $1$. At $\lambda =1$, the fitted coordinates along $u_1,u_2$ are $(27/10,1/2)$, retaining $9/10$ and $1/2$ of their ordinary least squares values $(3,1)$. At $\lambda =9$ they become $(3/2,1/10)$, retaining $1/2$ and $1/10$. The unfittable $u_3/2$ component contributes $1/4$ to every residual sum of squares, giving total values $1/4$, $59/100$, and $331/100$ at penalties $0,1,9$. These are constructed fits, not measured prediction results.

The penalty $\lambda$ is a hyperparameter, so fitting slopes at a fixed value and selecting that value are separate tasks. Use a validation set or K-fold cross-validation to compare candidate penalties by held-out squared-error loss. In each split, compute predictor sample means and scales from that split's training data, apply the same transformation to its validation set, and refit the slopes; using held-out information in preprocessing introduces data leakage. Minimum training residual sum of squares cannot select useful shrinkage: ordinary least squares already attains the smallest possible value.^[ref] After model selection, refit on all training data with the selected penalty. A separate test set or nested cross-validation assesses the entire selection procedure; a low selected validation score alone is no performance guarantee.^[ref]

Simple linear regression uses one predictor and an intercept, $Y_i=\alpha +\beta x_i+{\epsilon }_i$, with $\mathbb{E}({\epsilon }_i\mid x_i)=0$. The regression coefficients describe a straight conditional-mean line: $\alpha$ is its value at zero and $\beta$ its rate of change per unit of the predictor. When zero is outside the observed range, predictor centering can make the intercept refer to a more representative predictor value.

For fixed predictors and uncorrelated mean-zero errors of common variance ${\sigma }^2$, with $S_{xx}>0$,A larger predictor spread improves slope precision under this model. In the worked example $S_{xx}=10$ and $n=5$; the estimated error variance is $s^2=\mathrm{RSS}/(n-2)=0.8/3$. A normal linear model gives the slope pivot $(\hat{\beta }-\beta )/(s/\sqrt{S_{xx}})$ a Student t-distribution with $n-2$ degrees of freedom. The distinction between a fitted line and a reliable uncertainty statement is precisely the role of the error assumptions.

Universal kriging allows an unknown regression drift:The design matrix $X$ has full column rank, its entries and $x_0$ are known, and the regression coefficients $\beta$ are unknown. For example, rows $(1,s_i)$ and target row $(1,s_0)$ allow a linear spatial trend. Requiring $w^{T}Z$ to have zero bias for every $\beta$ gives $X^{T}w=x_0$. With known positive-definite matrix observation covariance matrix $\Sigma$, the minimizing weights solvewhere $\eta$ is a vector of Lagrange multipliers. Equivalently, the generalized least squares estimate and prediction areThe first term is the estimated trend alone; the second predicts the correlated regression residuals. With $q=x_0-X^T{\Sigma }^{-1}c$, the error variance isThe last, nonnegative term accounts for estimating the specified drift coefficients. Merely inserting a fitted drift into simple kriging and keeping its known-mean error variance would omit this term. For a Gaussian random field, the same expressions arise by integrating over drift coefficients in a diffuse prior distribution limit; this is an additional inferential interpretation, not an assumption required by the linear optimization.^[ref]