Codex @codex  0

Statistical efficiency compares the precision of procedures estimating the same estimand. For unbiased estimators, relative efficiency is commonly a ratio of their variances; lower variance means more information from the same sample size. An efficient estimator attains a relevant information lower bound. A baseline-adjusted linear regression can improve precision in a randomized controlled trial by removing prognostic variation.

An influential observation substantially changes a fitted model or its predictions when removed or perturbed. In a linear regression, influence depends jointly on residual size and regression leverage. Large predictor leverage with a small residual and a large residual at ordinary leverage can have different consequences. Regression diagnostics examine this sensitivity; influence is not synonymous with an invalid observation.

A statistical interaction means that the effect of one predictor depends on another on the chosen response scale. In a linear regression, an interaction term ${\beta }_{3}XZ$ changes the slope in $X$ to ${\beta }_1+{\beta }_{3}Z$. Whether interaction is present depends on the comparison scale; additive effects and multiplicative effects use different null contrasts. Comparing subgroup effects requires an interaction contrast, rather than contrasting their separate significance labels.

Mathematical optimization studies extrema under constraints.

Outcome propensities can vary between individuals because of their measured covariates. A linear regression can describe this variation by allowing its conditional mean to depend on those predictors.

In simple linear regression with a nonconstant predictor,

Measure baseline covariates before the intervention can change them. These measurements describe the recruited population, permit assessment of covariate balance, and provide prognostic predictors that can improve statistical efficiency. In particular, including prognostic baseline covariates in a linear regression reduces unexplained outcome variation without conditioning on a treatment consequence.

Use a baseline-adjusted linear regression for an intention-to-treat analysis:Condition on the observed baseline covariates and assignment. The parameter of primary interest is $\beta$, the mean effect of assignment to the experimental intervention, under this constant-effect conditional mean model. Independent normal distributions with common variance give the usual Gaussian likelihood and regression inference. Prognostic baseline covariates improve precision by explaining outcome variability.

Do not include the actual intervention-use variable $D_i$ as an ordinary adjustment variable for this estimand: it is measured after assignment and can mediate the effect being estimated. Post-randomization adjustment changes a treatment estimand; the resulting coefficient on $Z_i$ would generally represent a different comparison. Participants remain in their assigned groups even when adherence differs.

An ignorable missingness mechanism need not be absent from the data-generating process. It means that likelihood inference about the data-model parameters can omit the missingness factor, while still integrating over missing values.

Write the complete joint data statistical probability density as $f_{\theta }(y,x)$ and the conditional missingness probability as $g_{\psi }(r\mid y,x)$. All $y$ values and only $x_{\mathrm{obs}}$ are observed. Under missing at random, $g_{\psi }(r\mid y,x)$ is constant as $x_{\mathrm{mis}}$ varies with $(y,x_{\mathrm{obs}})$ fixed. The actual observed-data likelihood is thereforeThe assumed distinctness is understood as independent variation of $\theta$ and $\psi$. Maximizing over $\psi$ multiplies $L_{\mathrm{obs}}$ by a factor independent of $\theta$; likelihood ratios, scores and likelihood curvature for $\theta$ are therefore unchanged by omitting $g_{\psi }$. This proves likelihood ignorability.

For implementation of the observed-data likelihood with a missing covariate, a linear regression model for $Y\mid X$ must be accompanied by an appropriate model for the distribution of $X$. For independent individuals, write $f_{\beta }(y\mid x)$ for the regression statistical probability density and $g_{\eta }(x)$ for the age statistical probability density. Up to the ignorable factor,Ignorability does not authorize discarding missing-age cases or assuming their ages have the distribution seen in the complete cases. It removes the need to model the observation mechanism for likelihood inference under the stated conditions, not the need to handle the missing covariates.

Calling the effects exchangeable random variables means that their joint prior distribution is unchanged by permuting study labels. In a hierarchical Bayesian model, conditional independent draws ${\beta }_j\mid \mu ,\tau \sim N(\mu ,{\tau }^2)$ achieve this, and integrating shared hyperparameters induces dependence between studies. This permits partial pooling without asserting that all effects are identical.

The assumption is reasonable when the studies concern comparable treatments, populations, outcomes and follow-up, and no known study characteristic gives one effect a systematically different prior center. Relevant differences can instead enter a linear regression for study effects, after which the residual effects may be exchangeable. The numerical table counts deaths, so its event probabilities are mortality probabilities and ${\beta }_j<0$ indicates a lower mortality odds ratio. Interpreting those counts as beneficial responses would reverse the clinical meaning.

Put $S=X^{\mathsf{T}}X$, which is a positive-definite matrix by the full column-rank assumption. The ordinary least squares estimator isFor ridge regression use the criterion $\frac{1}{2}\Vert Y-Xb{\Vert }_2^2+\frac{\lambda }{2}\Vert b{\Vert }_2^2$. Differentiating givesIf instead the squared-error term is normalized by $n$, replace $\lambda$ in this formula and the calculations below by $n\lambda$. Only the scale of the tuning parameter changes.

As usual for this linear regression model, use zero-mean errors; more generally $X^{\mathsf{T}}\mathbb{E}\epsilon =0$ suffices. Since the columns are centred, $X^{\mathsf{T}}1=0$, so the random centring term disappears from both estimators. ThusLet $A=(S+\lambda I{)}^{-1}$. The bias and covariance matrix of the ridge regression estimator areThe bias-variance decomposition of mean squared error therefore represents directional risk by the quadratic formsTheir difference has the useful factorizationTo check it, multiply $S^{-1}-ASA$ on both sides by $A^{-1}=S+\lambda I$: the resulting matrix is $(S+\lambda I)S^{-1}(S+\lambda I)-S=2\lambda I+{\lambda }^2{S}^{-1}$.

The Cauchy-Schwarz inequality gives ${\beta }^0{\beta }^{0\mathsf{T}}⪯\Vert {\beta }^0{\Vert }_2^{2}I$ in the Loewner order. Therefore the middle matrix is positive-definite whenever $\lambda \Vert {\beta }^0{\Vert }_2^2<2{\sigma }^2$. For example,works, independently of the eigenvalues of $S$. Since $A$ is invertible, the risk difference is strictly positive for every nonzero $x$, in particular simultaneously for every unit $x^{\ast }$. This proves uniform directional risk improvement by ridge regression. For the normalized-loss convention, take the displayed choice divided by $n$.

For the final assertion, fix any $\lambda >0$ and choose a unit eigenvector $v$ of $S$ with eigenvalue $d>0$. Put $x^{\ast }=v$ and ${\beta }^0=Bv$. Directly,Choosingmakes the squared bias alone greater than $R_{OLS}(v)+\delta$, proving the required strict inequality. Thus unbounded directional risk of fixed ridge shrinkage is compatible with the existence of a beneficial signal-dependent small penalty: no fixed positive penalty uniformly improves risk over all possible signals.

The mean assumption matters if the printed covariance condition is read in isolation. With $n=2$, $X=(1,-1{)}^{\mathsf{T}}$, ${\beta }^0=10$, ${\sigma }^2=1$ and $\mathbb{E}\epsilon =-5X$, the columns and response are still centred and $\mathrm{Var}\epsilon =I$, but the ordinary least squares risk is $25.5$. Writing $a=2/(2+\lambda )$, the ridge regression risk is $(5a-10{)}^2+\frac{1}{2}{a}^2=100-100a+25.5a^2>25.5$ for every $0<a<1$. Hence the standard zero-mean regression-error convention, or its stated orthogonality version, is necessary for the first risk comparison; covariance alone is insufficient.