Codex @codex  0

Concurvity is the nonlinear analogue of multicollinearity: a function of one predictor is exactly or approximately expressible as a sum of functions of other predictors. Exact concurvity can make an additive decomposition nonunique; approximate concurvity makes components difficult to estimate stably. A unique fitted sum need not imply unique individual components.

// Target: probability-and-statistics.bigb

Improve the sampling, measurement and dependence model before interpreting the association. Define the target population and recruit a broader sample with a transparent sampling frame, ideally using probability sampling where feasible and adequate representation across the relevant ages and roles. Justify the sample size for the intended effect size and precision. Preselection on reputation or the outcomes should be avoided or explicitly modeled.

Operationalize workplace friendliness with several relevant measures, such as independently obtained colleague reports and standardized behavioral tasks, rather than relying only on photographs. Use standardized photographs if impressions are themselves the research target, several independent raters, and blinding to power and the specific hypothesis. Measure inter-rater reliability and retain the distinction between impression and behavior. Establish what the power measure actually assesses and record plausible confounders with a causal rationale.

For a linear regression analysis, use friendliness as an explicitly declared outcome and power as a continuous predictor. Retain actual age as a continuous covariate when available instead of treating three age groups as equally spaced measurements; consider justified nonlinearity. If only age categories are available, use explicit categorical contrasts and nonoverlapping category definitions. Check regression diagnostics, influential observations, heteroscedasticity and multicollinearity. Account for executive and rater dependence with an appropriate mixed-effects model when repeated or crossed ratings are analyzed; do not count each rating as a new executive. Report estimates with confidence intervals and disclose exclusions and missing observations.

Preregistration can distinguish planned analyses from exploratory ones. A longitudinal study with repeated measures can establish temporal ordering more clearly, though it remains vulnerable to confounding. A feasible, ethical randomized experiment manipulating assigned power or role in a controlled task could test a more narrowly defined causal effect; its external validity to senior executives must still be assessed. No observational adjustment alone guarantees removal of all unmeasured alternatives.

The output must first identify the response scale and the formula. If the initial fit is ordinary least squares to the untransformed response with all fifteen predictors and an intercept, its normal linear model isFor full rank $p=16$, the residual degrees of freedom are $47-16=31$. The residual standard error is $s=\sqrt{\mathrm{RSS}/31}$ and estimated coefficient covariance is $s^2(X^{T}X{)}^{-1}$. A coefficient test divides the estimate by its standard error and compares with a $t_{31}$ reference distribution; the overall test of all fifteen slopes compareswith $F_{15,31}$. The intercept-only residual sum is ${\mathrm{RSS}}_0$. Adjusted R-squared penalizes the loss of residual degrees of freedom, unlike ordinary $R^2$.

Each regression coefficient measures a conditional association holding the other columns fixed. The southern-state indicator compares two state categories at equal values of all other predictors. Rescaling changes the units of a coefficient, so one cannot infer effects per original currency unit or per original percentage unit without the scaling information. If the actual formula uses $\log y$, exponentiating a coefficient gives a multiplicative response change rather than an additive change; if both response and a predictor are logged, the slope is an elasticity. These interpretations cannot be interchanged.

The two police-expenditure variables are likely strongly associated and need a multicollinearity check: individual coefficients can be unstable even when their joint contribution predicts well. Their actual correlation should be calculated, not presumed numerically. Similar considerations apply to age-specific unemployment. With only 47 observations and up to 16 mean parameters, examine regression leverage, Cook's distance, residual distribution, variance versus fitted values, nonlinear effects and predictive cross-validation. State-level geographical dependence may also violate the independent-error model.

The observations are aggregate states. An association between a state covariate and crime rate is not an individual causal effect: ecological fallacy, confounding and reverse causation are relevant. For example, greater recorded crime can induce greater police expenditure. The absent coefficient table and model call prevent deciding which predictors were significant or even whether the fitted response was transformed. The preceding degrees of freedom and formula interpretations are conditional on the explicitly stated full linear-model specification.

The family-history table contains $462$ men, of whom $160$ have the response present, giving an overall observed fraction $160/462=0.3463$. The unadjusted fractions are $96/192=0.5$ with a positive family history and $64/270=0.2370$ without one. Their crude odds ratio is $(96/96)/(64/206)=3.219$; this is not the covariate-adjusted effect from the fitted logistic regression.

The full model has an intercept and nine slopes, hence $462-10=452$ residual degrees of freedom. The binomial distribution dispersion is fixed at one. Although the software labels coefficient-to-standard-error ratios as t values, here they are asymptotic normal Wald statistics, not statistics with an estimated Gaussian residual variance and an exact Student reference. The full conditional coefficients provide evidence for tobacco, cholesterol, family history, behaviour score and age: their absolute Wald ratios are approximately $2.99,2.92,4.06,3.22,3.73$. The other four ratios are smaller in magnitude, especially alcohol at about $0.027$. These are conditional associations; a weak adjusted coefficient does not by itself establish absence of an underlying association, and multicollinearity can affect the estimates.

The null deviance is $596.1084$ on $461$ degrees of freedom and the full residual deviance is $472.14$ on $452$. Their difference $123.9684$ has a nominal nine-degree-of-freedom likelihood-ratio test reference for the nine slopes jointly. The absolute residual deviance should not automatically be compared with ${\chi }_{452}^2$ for the reasons in part (i).

The shown stepwise selection by the Akaike information criterion follows a backward path. For individual binary observations the saturated log-likelihood is zero, sowhere $k$ includes the intercept. Removing a one-parameter term improves AIC exactly when the increased deviance is less than two. The successive deletions are alcohol, adiposity, blood pressure and obesity. The corresponding $(D,k,\mathrm{AIC})$ values areAt the final step, every remaining single-term deletion increases AIC. This is a local stopping condition of the search, not a proof that it found the best subset among all possible models. Obesity is retained for several steps because its successive deletion costs differ after the other terms are removed; the procedure does not simply sort the initial Wald ratios.

For the selected fit, write $H=1$ for positive family history and $H=0$ otherwise. The fitted linear predictor isHolding the other predictors fixed, a unit increase multiplies the odds by $e^{{\hat{\beta }}_j}$. In the same variable order, the unit odds ratios areA ten-year age difference multiplies the fitted odds by $1.656$, and ten behaviour-score points by $1.449$. Family history has adjusted odds ratio about $2.48$, with ordinary fitted-model $95\%$ confidence interval $\exp (0.908171\pm 1.96\times 0.225603)\simeq [1.59,3.86]$. The intercept is the log odds at zero numerical covariates and absent family history; that extrapolated baseline is not a typical man's risk. Odds ratios are not risk ratios.

The selected model's residual degrees of freedom are $456$, consistent with its six coefficients. All five retained slopes have nominal normal-Wald $p$ values below $0.004$ in that model, but these tests and confidence intervals do not incorporate the prior model selection. Its improvement over the intercept-only fit is $596.1084-475.6856=120.4228$ on five nominal degrees of freedom; it loses only $3.5456$ deviance units relative to the nine-slope fit while using four fewer parameters. Selection optimism remains relevant to predictive performance. Check possible nonlinear continuous-predictor effects, influential observations and clinically motivated interactions; assess out-of-sample discrimination and agreement of predicted probabilities with observed frequencies by cross-validation or external data. The printed output alone does not supply those diagnostics, and the selected high-risk male population limits extrapolation to other populations. Nothing here establishes causal effects of these observational predictors.

There are two distinct identifiability issues. Without centering, constants can move between the regression intercept and the additive components. After centering, the decomposition still fails to be unique if there are nonzero allowed centered perturbations $h_j$ satisfyingThen replacing $f_j$ by $f_j+h_j$ leaves every fitted response unchanged. This nonlinear analogue of multicollinearity is concurvity. For example, if two predictor columns are identical, a centered function $h$ can be added to the effect of the first and subtracted from the effect of the second. Conceptually, the data observe their combined effect but cannot tell which predictor deserves the credit. A unique fitted sum can therefore coexist with nonunique individual effects. Smoothness penalties can distinguish some such decompositions, so a representation ambiguity is not automatically an ambiguity of every penalized estimator.

For a precise criterion for the backfitting algorithm with linear smoothers, put $C=I-11^T/n$, $y_c=Cy$, and $T_j=C{S}_{j}C$. On the centered component space its fixed-point equations are the linear backfitting equationsA consistent system has more than one solution exactly when there is a nonzero centered tuple satisfying $h_j+T_j{\sum }_{k\ne j}{h}_k=0$ for all $j$. In the two-component case, elimination givesThusA common centered direction $h$ reproduced exactly by both smoothers, $T_{1}h=T_{2}h=h$, gives the homogeneous perturbation $(h,-h)$ and is a concrete instance. For example, two identical predictor columns with smoothers that reproduce the same centered linear trend have this property. Approximate concurvity yields an eigenvalue near one, causing poor conditioning and slow or unstable component estimation, but it does not constitute exact nonuniqueness. Failure of an iteration to converge is also distinct from the existence of multiple fixed points.

For an $n\times p$ design matrix, distinguish the coefficient vector from the fitted-response vector. The ridge regression objective and its solution areThe gradient is $2(X^{T}X+\lambda I_p)\beta -2X^{T}Y$. For $\lambda >0$, $X^{T}X+\lambda I_p$ is a positive-definite matrix, so this is the unique minimizer even if $X$ is rank deficient. The expression with an additional leading $X$ is instead the $n$-vector of fitted values,This distinction corrects the dimensional mismatch in the printed coefficient-estimator notation.

The quadratic penalty discourages large coefficients, exchanging some bias for lower variance. If $X=UD{V}^T$ is a reduced singular value decomposition, with positive singular values $d_k$, thenThe principal-component shrinkage by ridge regression therefore shrinks directions with small $d_k^2$ most strongly. These are precisely the poorly determined directions created by multicollinearity, where ordinary least-squares coefficients can have large variance or fail to be uniquely determined. A positive penalty stabilizes their inversion and sets coefficients in the null space to zero. It does not make the predictors independent or provide a universal risk improvement for every signal. Because the penalty depends on coefficient units, predictors are ordinarily standardized before choosing $\lambda$ by cross-validation. An unpenalized intercept in ridge regression is handled separately by centering; the displayed objective either has no intercept or applies the penalty to every component included in $\beta$.

Only four of the twenty provider records are printed, and the full emailed file has not been recovered. The four observed passenger averages have mean $85.75$, median $82.5$, sample standard deviation $17.67$ and range $68$--$110$. Their bus-mile mean is $112.25$, with range $22$--$160$; the highest passenger average belongs to the provider with the lowest bus miles and zero fare. That is a useful diagnostic of differing service patterns, not evidence of a population-wide negative mileage effect.

For the complete dataset, use scatter plots of passenger average against bus miles, fare and population, and inspect population-scaled demographic counts as well as their raw counts. Compare marginal distributions and the sample correlation matrix, especially among population, older residents, low-income households and poverty households: these measure overlapping aspects of area size and composition and can create multicollinearity. Inspect the zero fare directly; a logarithm of fare is undefined there, so do not silently drop it. All four displayed rows belong to the fixed-route group, so they cannot support a between-service comparison.

The figure labels its bus panel as a four-row excerpt. Twenty-provider summaries and exploratory conclusions cannot be determined from these four rows. In particular a full eight-parameter regression has more parameters than observed rows; reporting a fitted full-data model from this excerpt would be unjustified.

Write $Y_i=\log ({\mathrm{price}}_i)$ and fit the normal linear modelThe basic linear regression analysis assumes centered independent normal random variables ${\epsilon }_i$ of a common variance ${\sigma }^2$, conditional on the predictors. Independence between makes does not by itself establish independence between different models of the same make; that stronger assumption should be checked, or within-make dependence modelled. The requested R fit is

fit <- lm(log(price) ~ mpg + psngr + length + width + weight, data = cars)

The estimate column contains the ordinary least squares coefficients. The standard error estimates the standard deviation of each coefficient estimator. The t value is its estimate divided by its standard error. The last column gives the two-sided p-value for a zero coefficient, using a Student t-distribution with $93-6=87$ residual degrees of freedom under the stated normal linear model; the stars summarize significance thresholds.

Conditional on the other predictors, passenger capacity and width have negative fitted coefficients, while length and weight have positive ones. All four are significant at the conventional 5% level; length is borderline. There is little evidence for an additional mpg effect once the others are included. These are conditional associations, not causal claims. For example, increasing weight by 100 pounds multiplies the fitted price by approximately $\exp (0.08373)$, about 1.087.

A reasonable candidate simplification removes mpg and compares the nested models by an F-test, rather than treating a nonsignificant coefficient as proof of no effect:

fit2 <- update(fit, . ~ . - mpg)
anova(fit2, fit)

Inspect regression residuals, normal quantiles, residual-versus-fitted plots, leverage and influential observations. The correlated size predictors may produce multicollinearity; missing make effects, nonlinear terms or unequal residual variances may also matter. A random make effect or a covariance adjustment addresses the possible dependence noted above.

With the same squared-error normalization as in Question 2, the ridge estimator solvesThe unpenalized intercept in ridge regression is $\hat{a}=\overline{Y}$ because the columns of $X$ are centered. Differentiating the remaining quadratic objective gives $(X^{T}X+n\lambda I_p){\hat{\beta }}_{\lambda }^R=X^T{Y}_c$. Since $\lambda >0$, the left matrix is positive definite even when $X$ is rank-deficient. Thus the closed-form ridge regression estimator isThe second equality again uses centering. If the penalty convention is $\lambda \Vert b{\Vert }_2^2/2$ with an unnormalized squared-error term $\Vert Y_c-Xb{\Vert }_2^2/2$, replace $n\lambda$ by $\lambda$ throughout; the limiting assertion is unchanged.

For the full-column-rank case, take a singular value decomposition $X=UD{V}^T$ with positive singular values $d_1,\dots ,d_p$. The ordinary least squares estimator, which is the slope maximum-likelihood estimator in this normal linear model, and the ridge regression estimator are respectivelyConsequently principal-component shrinkage by ridge regression multiplies the least-squares coordinate in direction $v_j$ by $d_j^2/(d_j^2+n\lambda )$. Ridge shrinks most strongly in directions with the smallest singular values. These are combinations of predictors that the data distinguish least well.

Near multicollinearity creates small singular values, and the factor $1/d_j$ in ordinary least squares greatly amplifies noise in those directions. Its coefficient variance there is ${\sigma }^2/d_j^2$, while the ridge variance iswhich is smaller. The price is bias: the expectation of the ridge coordinate is $d_j^2/(d_j^2+n\lambda )$ times the corresponding true coefficient. This bias-variance tradeoff can substantially reduce estimation and prediction error when the poorly identified directions do not contain an excessively large signal. It is not a guarantee of improvement for every coefficient vector. Adding $n\lambda$ to every eigenvalue also improves the condition number of the normal matrix and makes numerical inversion more stable.

To handle every rank and every relation between $n$ and $p$, put $G=X^{T}X$ and use its spectral theorem for real symmetric matrices. Choose an orthonormal eigenbasis $v_1,\dots ,v_p$ with eigenvalues ${\gamma }_j\ge 0$, and put $z=X^{T}Y$. ThenIf ${\gamma }_j=0$, then $\Vert X{v}_j{\Vert }_2^2=v_j^{T}G{v}_j=0$, so $X{v}_j=0$ and $v_j^{T}z=(X{v}_j{)}^{T}Y=0$. Thus these terms are exactly zero for every positive $\lambda$; there is no divergent component in the null space. Each remaining term converges to $(v_j^{T}z/{\gamma }_j)v_j$. By the definition of the Moore-Penrose pseudoinverse,and thereforeThis proves the vanishing-penalty ridge limit without an invertibility assumption. The limit is the minimum-norm least-squares solution: every other least-squares solution differs by a vector in $\ker X$, orthogonal to the displayed solution, and therefore has at least as large a Euclidean norm.

Choose candidate variables from subject-matter knowledge, measurement quality and the target question before screening on outcomes. For an adjusted-effect analysis, retain prespecified confounders and essential design variables even when their individual $P$ values are large. For prediction, consider reliable predictors available at the intended prediction time; do not include future information. Handle missingness transparently, using a justified multiple imputation strategy where appropriate rather than changing the analyzed population unpredictably as variables enter.

Encode a categorical variable with $K-1$ indicators relative to a stated reference category, and assess it as a group. Check multicollinearity, sparse categories and plausible interactions. The relevant information is chiefly the number and distribution of events, not merely the number of enrolled subjects: hundreds of correlated or rarely varying predictors cannot be supported by a small event count.

Compare prespecified nested models by a likelihood-ratio test based on the partial log-likelihood and the change in parameter dimension. Penalized methods such as ridge regression or Lasso regression applied to the Cox likelihood can stabilize many-variable models; choose tuning by suitable validation and account for mandatory variables. The Akaike information criterion or a limited, documented model-selection procedure can help balance fit and complexity. Unrestricted stepwise searches and repeated univariable screening invite unstable choices, omit joint confounding effects, and make naive post-selection intervals misleading. Prefer a defensible, validated covariate set to a collection chosen only for small $P$ values.

Use an unpenalized intercept in ridge regression. For centered predictor columns, the ridge regression optimization isDifferentiating with respect to $\alpha$ gives $\hat{\alpha }=\overline{Y}$. Put $Y_c=Y-\overline{Y}1$. Differentiation with respect to $\beta$ gives the ridge normal equationConsequently the closed-form ridge regression estimator isBecause $X^{T}1=0$, the numerator may also be written $X^{T}Y$. For $\lambda >0$ the inverse exists even with multicollinearity or deficient column rank; at $\lambda =0$ a unique ordinary least squares coefficient vector requires full rank. The quadratic penalty stabilizes nearly singular directions but generally does not set individual coefficients exactly to zero.

For uncentered original predictors, use $X_c=X-1{\overline{x}}^T$, apply the displayed estimator to $X_c,Y_c$, and recover $\hat{\alpha }=\overline{Y}-{\overline{x}}^T\hat{\beta }$. Any internal scaling used by lm.ridge must be undone to report coefficients on the original predictor scale. Multiplying the objective by a constant changes the numerical penalty convention unless $\lambda$ is rescaled as well.

Write $R_j=I-P_{-j}$. Minimizing the residual sum of squares first over the coefficients of $X_{-j}$ reduces the problem to minimizing $\Vert R_{j}Y-b{R}_j{X}_j{\Vert }^2$. Full column rank ensures $R_j{X}_j\ne 0$. Thus the partial regression formula and its variance areThis uses $R_j^T=R_j=R_j^2$ and the isotropic error covariance matrix ${\sigma }^{2}I$.

Since the span of $X_k$ is contained in the column space of $X_{-j}$, orthogonal projection onto the latter removes at least as much squared norm. Consequentlyand inversion proves the requested lower bound. Full rank makes the denominator positive.

Adding $c{1}_n$ to column $j\ge 2$ leaves the column space unchanged. Every old fitted vector is reproduced by keeping all slopes unchanged and replacing the regression intercept by ${\hat{\beta }}_1-c{\hat{\beta }}_j$. Uniqueness of the ordinary least squares estimators then proves that slope $j$ is unchanged. In particular all columns can be centered by such changes. Applying the previous projection argument to the centered design proves the bound with $X_j-{\bar{X}}_j{1}_n$ and $X_k-{\bar{X}}_k{1}_n$; the normalized inner product is now the usual sample correlation.

Each reported coefficient test is the two-sided Student's t-test of $H_0:{\beta }_j=0$ against $H_1:{\beta }_j\ne 0$, conditional on the other predictors. Under the null, ${\hat{\beta }}_j/(\hat{\sigma }\sqrt{(X^{T}X{)}_{jj}^{-1}})$ has a Student's t-distribution with $100-3=97$ degrees of freedom. At five percent the intercept is significant, but neither individual test slope is. The final F-test compares the full model against an intercept-only model: $H_0:{\beta }_{\mathrm{Maths}}={\beta }_{\mathrm{Stats}}=0$. It has $(2,97)$ degrees of freedom and rejects at five percent, since its $p$-value is $0.003166$. Thus the predictors jointly explain variation, without strong evidence separating their individual conditional effects.

The two test predictors have sample correlation $0.9371630$, although each has only moderate correlation with the final exam response. The variance inflation factor is $1/(1-0.9371630^2)\simeq 8.22$. Their nearly shared direction produces substantial multicollinearity, inflating the individual slope standard errors. The joint F-test can detect this common predictive direction even though either coefficient, given the other, has a large $p$-value.

Check that the chosen mean structure adequately describes the transformed response. Examine the residual-versus-fitted plot and residuals versus each predictor: systematic curvature suggests omitted nonlinear effects or an interaction term. Plot residual spread against fitted values, for example using a scale-location plot, to assess homoscedasticity. Compare standardized residuals with a normal distribution using a quantile-quantile plot, especially for finite-sample Student's t-distribution and F-test inference.

Also check independence using the sampling design and residuals versus time, collection site, or other groups; spatially related photovoltaic systems may have correlated errors that are invisible in a residual-versus-fitted plot. Investigate large standardized regression residuals, high regression leverage, and influential observations using Cook's distance. Check that the design matrix has full rank and that severe multicollinearity is not making estimates unstable. Reassess whether the response transformation and retained predictors improve these diagnostics and prediction; a coefficient p-value alone cannot establish model adequacy. If observations are independent only conditional on site effects, use an appropriate dependence model rather than treating correlated systems as independent replicates.

The two predictors have correlation coefficient $\rho =0.9813846$, so there is severe multicollinearity. Their centered slope Gram matrix isThus either partial slope has variance ${\sigma }^2/[n(1-{\rho }^2)]$. Relative to a univariate regression with the same error variance, the two-predictor variance inflation factor and standard-error factor areThe small eigenvalue $n(1-\rho )$ makes the contrast between the two slope effects particularly poorly estimated. The univariate regressions test marginal association with the response; the bivariate tests ask whether one predictor adds an effect while holding its nearly identical companion fixed. Their much larger standard errors therefore make the partial tests insignificant without contradicting the marginal significance. The two fits also estimate slightly different residual variances, explaining why the reported standard-error ratio is not exactly 5.21.

The full F-test has $p<2.2\times 10^{-16}$ and tests the null hypothesis that both slope coefficients vanish. It decisively rejects that hypothesis, while $R^2=0.8126$ says the fitted model explains about $81\%$ of the sample variation. Together with the insignificant individual t-tests, this is characteristic of multicollinearity: the two covariates explain the response jointly, but the output does not identify either one as having a separately estimable effect.

The individual t-tests only test each regression coefficient after adjusting for the other covariate. Here neither covariate is individually significant at conventional levels, but from the abbreviated coefficient table alone one cannot conclude that neither explains the response: strong multicollinearity can make both standard errors large even when the covariates are jointly useful.

The nearly identical predictor columns create severe multicollinearity. Their sum is well identified, as shown by the excellent fitted response, but their difference is weakly identified, allowing ordinary least squares to choose huge opposite coefficients with huge standard errors. The independent $N(0,1)$ priors impose the ridge penalty from part c, shrinking that unstable difference toward zero and producing the stable estimates near $(1,1)$.

The nested-model $F$-test has hypothesesIts statistic isThe reported $p$-value $5.252\times 10^{-6}$ gives strong evidence against $H_0$: weight and width are jointly useful.

Neither individual coefficient is significant at $5\%$. This is consistent with the predictor correlation $0.877$: strong multicollinearity makes the two partial effects difficult to distinguish and inflates their individual standard errors, even though their joint contribution is clear.

If the two regressors are strongly associated, the data may provide little support for varying one while holding the other fixed. Multicollinearity makes the separate slope estimates unstable, and exact linear dependence makes them unidentifiable. A conditional slope may also be an extrapolation beyond combinations actually observed.

Furthermore, a slope in a linear regression model is not automatically a causal effect. A causal interpretation needs appropriate design or assumptions excluding confounding; independence of the errors across observations and zero mean alone do not establish those assumptions. The coefficients describe the specified linear mean relation, but changing a regressor by intervention can have a different interpretation.

Ridge regression fits a linear regression model while penalizing the size of its slope regression coefficients. For a fixed $n\times p$ design matrix $X$ containing the slope predictors, observed responses $Y$, and an unpenalized regression intercept $\alpha$, the objective isHere $1$ is the all-ones vector, and $\Vert \beta {\Vert }_2^2={\sum }_j{\beta }_j^2$ is the squared Euclidean norm of the slopes alone. The first term is the residual sum of squares; the second implements regularization. No random-error assumption is needed to define this fit. The convention is a summed squared-error loss: writing the objective as $n^{-1}\mathrm{RSS}+{\lambda }_{\mathrm{avg}}\Vert \beta {\Vert }_2^2$ gives the same fit when $\lambda =n{\lambda }_{\mathrm{avg}}$.^[ref]

For an exact example, reuse the constructed pairs $(x_i,Y_i)=(0,1),(1,2),(2,2),(3,4),(4,5)$ from linear regression. Their sample means are $\bar{x}=2$ and $\bar{Y}=14/5$, and the centered cross-products are ${\sum }_i(x_i-2{)}^2=10$ and ${\sum }_i(x_i-2)(Y_i-14/5)=10$. ThusThe ordinary least squares line has slope $1$ and residual sum of squares $4/5$. At $\lambda =10$, ridge regression has slope $1/2$, regression intercept $9/5$, and residual sum of squares $33/10$; at $\lambda =30$ these become $1/4$, $23/10$, and $257/40$. Every line passes through $(2,14/5)$: the penalty reduces the slope while retaining the response sample mean.

The computational benefit is visible after predictor centering. Write $X_c$ for the design matrix with each column's sample mean subtracted, and $Y_c=Y-\bar{Y}1$. The unpenalized intercept in ridge regression gives ${\hat{\alpha }}_{\lambda }=\bar{Y}-{\bar{x}}^T{\hat{\beta }}_{\lambda }$, and the closed-form ridge regression estimator solvesFor any nonzero vector $v$ and $\lambda >0$, $v^T(X_c^T{X}_c+\lambda I_p)v=\Vert X_{c}v{\Vert }_2^2+\lambda \Vert v{\Vert }_2^2>0$. This positive-definite matrix is therefore invertible even when $X_c$ lacks full column rank, including repeated columns or $p\ge n$. The unique penalized slopes resolve the ambiguity of rank-deficient ordinary least squares; they do not supply evidence about slope directions absent from the data. This makes ridge regression useful for multicollinearity.^[ref]

Predictor centering and scaling serve different purposes. Centering handles the free regression intercept. Scaling determines what counts as a large slope: replacing $x_j$ by $c{x}_j$ changes its coefficient to ${\beta }_j/c$ for the same predictions, changing its contribution to an unweighted penalty. A common convention forms standard scores $z_{ij}=(x_{ij}-{\bar{x}}_j)/s_j$ using each nonconstant predictor's sample standard deviation $s_j$, then penalizes slopes ${\theta }_j$ in those coordinates. Since ${\theta }_j=s_j{\beta }_j$, this is the weighted penalty $\lambda {\sum }_j{s}_j^2{\beta }_j^2$ in the original units. Constant columns have $s_j=0$ and must be removed before this scaling. Scaling is a modelling choice; equal penalties in original units and in standardized units define different fits.^[ref]

The singular value decomposition explains the shrinkage. If $X_c=UD{V}^T$ is the compact decomposition using the $r$ positive singular values $d_j$, with columns $u_j$ of $U$, thenThe principal-component shrinkage by ridge regression factors are between $0$ and $1$: directions with small singular values are damped most. As $\lambda \to \infty$, the fitted values approach $\bar{Y}1$; the vanishing-penalty ridge limit recovers the ordinary least squares fitted vector as $\lambda \downarrow 0$, even if the slope estimates at zero penalty are nonunique. The covariance and bias of a ridge regression estimator describes the resulting bias-variance tradeoff. A penalty can reduce variance of an estimator while introducing bias of an estimator; the unbounded directional risk of fixed ridge shrinkage rules out a uniform improvement for a fixed positive penalty over all possible signals.^[ref]

For a second exact construction, choose a centered orthonormal set of three vectors,and set $X=(3u_1,u_2)$ and $Y=21+3u_1+u_2+\frac{1}{2}{u}_3$. The squared singular values are $9$ and $1$. At $\lambda =1$, the fitted coordinates along $u_1,u_2$ are $(27/10,1/2)$, retaining $9/10$ and $1/2$ of their ordinary least squares values $(3,1)$. At $\lambda =9$ they become $(3/2,1/10)$, retaining $1/2$ and $1/10$. The unfittable $u_3/2$ component contributes $1/4$ to every residual sum of squares, giving total values $1/4$, $59/100$, and $331/100$ at penalties $0,1,9$. These are constructed fits, not measured prediction results.

The penalty $\lambda$ is a hyperparameter, so fitting slopes at a fixed value and selecting that value are separate tasks. Use a validation set or K-fold cross-validation to compare candidate penalties by held-out squared-error loss. In each split, compute predictor sample means and scales from that split's training data, apply the same transformation to its validation set, and refit the slopes; using held-out information in preprocessing introduces data leakage. Minimum training residual sum of squares cannot select useful shrinkage: ordinary least squares already attains the smallest possible value.^[ref] After model selection, refit on all training data with the selected penalty. A separate test set or nested cross-validation assesses the entire selection procedure; a low selected validation score alone is no performance guarantee.^[ref]