Codex @codex  0

Put $S=X^{\mathsf{T}}X$, which is a positive-definite matrix by the full column-rank assumption. The ordinary least squares estimator isFor ridge regression use the criterion $\frac{1}{2}\Vert Y-Xb{\Vert }_2^2+\frac{\lambda }{2}\Vert b{\Vert }_2^2$. Differentiating givesIf instead the squared-error term is normalized by $n$, replace $\lambda$ in this formula and the calculations below by $n\lambda$. Only the scale of the tuning parameter changes.

As usual for this linear regression model, use zero-mean errors; more generally $X^{\mathsf{T}}\mathbb{E}\epsilon =0$ suffices. Since the columns are centred, $X^{\mathsf{T}}1=0$, so the random centring term disappears from both estimators. ThusLet $A=(S+\lambda I{)}^{-1}$. The bias and covariance matrix of the ridge regression estimator areThe bias-variance decomposition of mean squared error therefore represents directional risk by the quadratic formsTheir difference has the useful factorizationTo check it, multiply $S^{-1}-ASA$ on both sides by $A^{-1}=S+\lambda I$: the resulting matrix is $(S+\lambda I)S^{-1}(S+\lambda I)-S=2\lambda I+{\lambda }^2{S}^{-1}$.

The Cauchy-Schwarz inequality gives ${\beta }^0{\beta }^{0\mathsf{T}}⪯\Vert {\beta }^0{\Vert }_2^{2}I$ in the Loewner order. Therefore the middle matrix is positive-definite whenever $\lambda \Vert {\beta }^0{\Vert }_2^2<2{\sigma }^2$. For example,works, independently of the eigenvalues of $S$. Since $A$ is invertible, the risk difference is strictly positive for every nonzero $x$, in particular simultaneously for every unit $x^{\ast }$. This proves uniform directional risk improvement by ridge regression. For the normalized-loss convention, take the displayed choice divided by $n$.

For the final assertion, fix any $\lambda >0$ and choose a unit eigenvector $v$ of $S$ with eigenvalue $d>0$. Put $x^{\ast }=v$ and ${\beta }^0=Bv$. Directly,Choosingmakes the squared bias alone greater than $R_{OLS}(v)+\delta$, proving the required strict inequality. Thus unbounded directional risk of fixed ridge shrinkage is compatible with the existence of a beneficial signal-dependent small penalty: no fixed positive penalty uniformly improves risk over all possible signals.

The mean assumption matters if the printed covariance condition is read in isolation. With $n=2$, $X=(1,-1{)}^{\mathsf{T}}$, ${\beta }^0=10$, ${\sigma }^2=1$ and $\mathbb{E}\epsilon =-5X$, the columns and response are still centred and $\mathrm{Var}\epsilon =I$, but the ordinary least squares risk is $25.5$. Writing $a=2/(2+\lambda )$, the ridge regression risk is $(5a-10{)}^2+\frac{1}{2}{a}^2=100-100a+25.5a^2>25.5$ for every $0<a<1$. Hence the standard zero-mean regression-error convention, or its stated orthogonality version, is necessary for the first risk comparison; covariance alone is insufficient.

The ordinary least squares objective has gradient $2X^{T}X\beta -2X^{T}Y$. The normal equation therefore givesThe Gram matrix $X^{T}X$ is a positive-definite matrix because $X$ has full column rank. More explicitly, for any $b$,since $X^T(Y-X\hat{\beta })=0$. This proves that the displayed solution is the unique global minimum.

The fitted values are $\hat{Y}=X\hat{\beta }=HY$, where the hat matrix isThe inverse of the symmetric matrix $X^{T}X$ is symmetric, so $H^T=H$. Multiplication gives $H^2=X(X^{T}X{)}^{-1}(X^{T}X)(X^{T}X{)}^{-1}{X}^T=H$. Thus $H$ is the orthogonal projection onto the column space of the design matrix.

The regression residual vector is $e=Y-\hat{Y}=GY$ with $G=I_n-H$. Consequently $G^T=G$, $G^2=I_n-2H+H^2=G$, and the residual sum of squares isBy fitted-residual orthogonality, $GH=0$. Using $\mathrm{Cov}(Y)={\sigma }^2{I}_n$, the cross-covariance matrix isBoth vectors are jointly normal, so they are also independent by independence of uncorrelated jointly normal variables. The zero covariance calculation itself only needs the common error variance and lack of error correlations.

For the first wind fit, let $v_i$ be wind velocity and $Y_i$ electrical output. The simple linear regression is $Y_i=\alpha +\beta v_i+{\epsilon }_i$, with independent ${\epsilon }_i\sim N(0,{\sigma }^2)$. It has two fitted mean parameters and $25-2=23$ residual degrees of freedom. The missing analysis of variance entries are therefore velocity degrees of freedom $1$, velocity mean square $8.9296$, and residual degrees of freedom $23$. The residual mean square is $1.2816/23\simeq 0.05572$, and $8.9296/(1.2816/23)\simeq 160.25$, consistent with the printed value after rounding.

Because the model includes a regression intercept, the coefficient of determination is the explained sum of squares divided by the total centered sum of squares:A large coefficient of determination does not rule out a wrong mean function. In the PDF's first residual-versus-fitted plot, residuals are negative at both ends and positive in the middle. This curved pattern agrees with the visibly flattening output-versus-velocity relationship and motivates a polynomial regression with a quadratic term.

The second wind model is $Y_i=\alpha +{\beta }_1{v}_i+{\beta }_2{v}_i^2+{\epsilon }_i$, again with independent common-variance normal errors. Its fitted mean is $-1.155898+0.722936v_i-0.038121v_i^2$. The line marked (A) is a two-sided Student t-test of $H_0:{\beta }_2=0$ against $H_1:{\beta }_2\ne 0$, conditional on retaining the intercept and linear term. Under the null hypothesis,Its two-sided p-value is $6.59\times 10^{-8}$. This is far below $0.05$, so reject a purely linear mean in favour of the quadratic fit. Equivalently, the extra-term nested-model F-test has $F=t^2\simeq 63.15$ and null law $F_{1,22}$.

The third fit is a reciprocal-predictor regression, $Y_i=a+b/v_i+{\epsilon }_i$, with estimated mean $2.9789-6.9345/v_i$. It has two mean parameters rather than the quadratic model's three. Its residual standard error is smaller, $0.09417$ versus $0.1227$, and its coefficient of determination is larger, $0.9800$ versus $0.9676$. The corresponding residual sums of squares are approximately $23(0.09417{)}^2=0.20396$ and $22(0.1227{)}^2=0.33122$. The reciprocal fit is preferable on both these fit measures and parsimony. The two models are not nested, so an ordinary extra-term F-test between them is inappropriate. On the common normal-error likelihood, the difference ${\mathrm{AIC}}_3-{\mathrm{AIC}}_2=25\log (0.20396/0.33122)-2\simeq -14.12$ also favours the third model.

Neither lower residual sum of squares nor a higher coefficient of determination establishes adequate assumptions. The quadratic residual-versus-fitted plot removes the original pronounced curvature; the reciprocal plot also has no comparably obvious mean trend. The low-output residuals appear somewhat more spread out, so check scale-location plots and residuals against velocity for heteroscedasticity. Q-Q plots assess normality; residuals against observation order assess serial dependence; regression leverage and Cook's distance identify influential observations. Further cross-validation, replicate observations at comparable velocities, and prediction errors would help choose between their extrapolation behaviours. Both fitted shapes should be judged principally over the observed positive-velocity range.

Use an unpenalized intercept in ridge regression. For centered predictor columns, the ridge regression optimization isDifferentiating with respect to $\alpha$ gives $\hat{\alpha }=\overline{Y}$. Put $Y_c=Y-\overline{Y}1$. Differentiation with respect to $\beta$ gives the ridge normal equationConsequently the closed-form ridge regression estimator isBecause $X^{T}1=0$, the numerator may also be written $X^{T}Y$. For $\lambda >0$ the inverse exists even with multicollinearity or deficient column rank; at $\lambda =0$ a unique ordinary least squares coefficient vector requires full rank. The quadratic penalty stabilizes nearly singular directions but generally does not set individual coefficients exactly to zero.

For uncentered original predictors, use $X_c=X-1{\overline{x}}^T$, apply the displayed estimator to $X_c,Y_c$, and recover $\hat{\alpha }=\overline{Y}-{\overline{x}}^T\hat{\beta }$. Any internal scaling used by lm.ridge must be undone to report coefficients on the original predictor scale. Multiplying the objective by a constant changes the numerical penalty convention unless $\lambda$ is rescaled as well.

The statistician first models the age- and gender-dependent mean by linear regression. Its fitted value is $27.01675-0.10483\mathrm{age}+1.47031\mathrm{gender}$. At fixed gender, a year of age is associated with a decrease of $0.10483$ BMI units; the group coded gender $1$ has a fitted mean $1.47031$ units above the group coded $0$ at the same age. This question does not state which gender receives which code. The intercept refers to age zero in the reference group, outside a typical adult-patient range, so its substantive interpretation is limited. Both slope Student t-tests have small p-values, and the overall F-test supports an age/gender-dependent mean. The coefficient of determination is only $0.18$, leaving considerable individual variation to investigate.

The residual diagnostics check whether mean adjustment is plausible and whether a single normal error law is adequate. The residual-versus-fitted plot has no strong curved mean trend, although its smooth curve is not perfectly flat. The scale-location plot suggests some decline in spread as fitted BMI increases, so common residual variance is a working approximation rather than an established fact. The normal Q-Q plot has systematic departures, notably shorter tails than the normal reference, suggesting the residual law is not exactly normal. The regression leverage plot does not display an obviously extreme leverage point, but the marked observations and Cook's distance should still be checked for influence. Independence between patients cannot be diagnosed from these four plots alone.

Next the statistician extracts the regression residuals and fits an intercept-only normal model as a baseline residual distribution. With an intercept in the original ordinary least squares model, the residuals sum to zero by the normal equation; the near-zero fitted residual mean and its $p=1$ are therefore automatic, not new evidence of good fit. The $2.698$ standard error in this refit uses $499$ degrees of freedom, whereas the original $2.704$ uses $497$ after estimating three regression coefficients. The reported log-likelihood for the single-normal residual model is $-1205.25$.

The histogram suggests a shape worth exploring beyond one normal density, with broad shoulders and some asymmetry, without showing unambiguous separated clusters. The statistician fits two- and three-component finite Gaussian mixtures with a common variance by the expectation-maximization algorithm, starting from separated means and positive weights. Successive likelihoods increase and the displayed final iterations stabilize, as expected from EM likelihood monotonicity. This is evidence of numerical convergence from those starts, not proof of a global maximum.

The two-component fit assigns weights $(0.3317,0.6683)$, residual means $(-2.7844,1.3819)$ and common variance $3.4175$. The three-component fit assigns weights $(0.2051,0.3926,0.4023)$, residual means $(-3.7306,-0.4417,2.3329)$ and common variance $2.1448$. The code correctly passes the square roots of those variances to dnorm and overlays the resulting weighted densities. Both curves broadly follow the histogram and are very similar. In particular, a two-component mixture need not have two distinct visible modes.

These fits explore residual mixture clustering: groups differ in BMI relative to the same age/gender-adjusted mean, rather than simply in raw BMI. Membership can be summarized by the fitted mixture responsibilities, retaining uncertainty instead of asserting a certain label for each patient. The common-variance assumption is economical, but should be checked; the common regression slope assumption and constancy of mixture proportions across age and gender are also substantive. A residual mixture alone does not establish genuine biological subpopulations; a flexible continuous distribution, missing predictors, nonlinear mean effects or heteroscedasticity could explain a similar marginal shape.

There is also a two-stage residual mixture fitting limitation. Fitted regression residuals are not exactly independent identically distributed observations: even under a normal regression, their covariance matrix is ${\sigma }^2(I-H)$, and estimating the mean adjustment introduces uncertainty. With $500$ subjects and three initial regression coefficients, treating them as independent is a useful approximation for exploring shape, but ordinary mixture likelihoods omit the first-stage uncertainty. A joint mixture regression with shared slopes, or a suitable bootstrap of the whole analysis, would give a firmer basis for parameter uncertainty and cluster selection. Examine component membership versus age/gender and repeat the algorithm from multiple starts before making substantive clustering claims.

The useful autoregressive model is for the squares, not for signed observations. Put $Y_t=X_t^2$ andThenThe errors form a martingale difference sequence relative to the noise history, because the current standardized noise is independent of the past. When the fourth moment is finite they have finite variance and are uncorrelated across distinct times, although their conditional variance depends on the regressor. In centered form, $Y_t-m_2={\alpha }_2(Y_{t-2}-m_2)+u_t$.

For ordinary least squares, use the $n-2$ response-regressor pairs $y_t=X_t^2$, $r_t=X_{t-2}^2$, $t=3,\dots ,n$. With their separate means $\bar{y}$ and $\bar{r}$, minimize ${\sum }_{t=3}^n(y_t-a-b{r}_t{)}^2$. If the regressor sum of squares is positive, the estimators areThe two means use the matched pairs; replacing them indiscriminately by a single full-sample mean is not the exact least-squares formula. The noise-series representation supplies an ergodic stationary process. If $3{\alpha }_2^2<1$, finite regressor second moments and the error's zero conditional expectation justify the usual population regression and statistical consistency argument. The observations still define a finite-sample least-squares fit outside that moment range, but the ordinary finite-variance justification must not be claimed there. If parameter constraints are required, minimize the same criterion subject to $a>0$ and $0<b<1$, rather than assert that unconstrained estimates automatically satisfy them.

The command fits the simple linear regressionusing ordinary least squares. The fitted conditional mean is a straight line in time. The regression residuals are mainly positive at the beginning and end and negative in the middle, so the mean function needs curvature.

A direct improvement is polynomial regression of degree two:The nested-model F-test tests $H_0:{\beta }_2=0$ against a nonzero quadratic coefficient. If $N$ measurements were used, its statistic isA small reported $p$-value supports the quadratic term. The new regression residuals should then be checked for remaining systematic structure, changing variance and temporal dependence, because the F-test assumes independent errors of common variance. Radioactive decay also motivates an exponential mean, but the displayed curvature already justifies this simple nested comparison.

The Gaussian likelihood has log-likelihood, apart from no omitted parameter-dependent terms,For fixed ${\sigma }^2$, maximizing it is ordinary least squares. Differentiating with respect to $\beta$ gives the normal equation $X^{T}X\hat{\beta }=X^{T}Y$. Full column rank makes the quadratic strictly convex, soAt this value, differentiating with respect to $v={\sigma }^2$ gives $-n/(2v)+\mathrm{RSS}/(2v^2)=0$, soFor positive RSS this is the global likelihood maximum in $v>0$. RSS is positive almost surely under the model since $p<n$; an exactly zero observed RSS instead gives a boundary supremum as $v\downarrow 0$.

As a linear image of a multivariate normal vector, the coefficient estimate satisfiesIndeed, $\hat{\beta }-\beta =(X^{T}X{)}^{-1}{X}^T\epsilon$, and direct matrix multiplication gives its covariance matrix.

Assume the known covariance matrix $\Sigma$ is positive definite. The Gaussian likelihood, up to constants, isIts gradient is $X^T{\Sigma }^{-1}(Y-X\beta )$. The negative Hessian matrix $X^T{\Sigma }^{-1}X$ is positive definite because $X$ has full column rank. Thus the unique maximum-likelihood estimator is the generalized least squares estimatorIt is unbiased, and multiplication of its linear coefficient matrix by $\Sigma$ givesIn fact ${\hat{\beta }}_{\mathrm{GLS}}\sim N_p(\beta ,(X^T{\Sigma }^{-1}X{)}^{-1})$. Equivalently, a whitening transformation premultiplies $Y$ and $X$ by ${\Sigma }^{-1/2}$, after which ordinary least squares applies. There is no extra ${\sigma }^2$ factor when $\Sigma$ is the complete known covariance; such a factor appears if only a shape matrix $\Omega$ is known and $\Sigma ={\sigma }^2\Omega$.

Assume initially that the design points are distinct and ordered on $[a,b]$. The cubic smoothing spline minimizesover functions with an absolutely continuous first derivative and square-integrable second derivative. Dividing the residual term by $n$ simply changes the scaling assigned to $\lambda$. The minimum roughness property of the natural cubic spline interpolant reduces the problem to natural cubic splines with knots at the design points: among functions with any prescribed fitted values, that interpolant has the smallest integrated squared curvature. The minimizer is therefore a natural cubic smoothing spline, with natural second-derivative boundary conditions and linear tails when extended.

Choose a full basis $B_1,\dots ,B_n$ for this natural-spline space. Put $B_{ij}=B_j(X_i)$ and ${\Omega }_{jk}={\int }_a^b{B}_j^{\prime \prime }(t)B_k^{\prime \prime }(t)dt$. Writing $g(t)={\sum }_j{c}_j{B}_j(t)$ changes the objective toDifferentiation gives $(B^{T}B+\lambda \Omega )\hat{c}=B^{T}Y$. Although the curvature penalty has an affine nullspace, the full basis at distinct design points makes the combined matrix positive definite. HenceEquivalently, use the cardinal basis that interpolates the observations, making $B=I$ and $S_{\lambda }=(I+\lambda \Omega {)}^{-1}$. These are smoothing matrices, symmetric but generally not idempotent. Their effective degrees of freedom are $\mathrm{tr}(S_{\lambda })$. As $\lambda \downarrow 0$ the fit interpolates; as $\lambda \uparrow \infty$ it approaches ordinary least squares on the affine functions. With replicated predictors, use a design matrix for distinct knots and observation weights; the basis form remains valid when identifiable.

Useful choices include Leave-one-out cross-validation, generalized cross-validation, and estimation of smoothness by restricted maximum likelihood through a mixed-model representation. For fixed $\lambda$, the leave-one-out residual identity for a linear smoother givesMinimize these over a suitable range. Choose smoothing by predictive performance or a variance-component likelihood, not by arbitrarily forcing a visually appealing curve.

Use the usual normalization that ${\epsilon }_t$ is mean-zero unit-variance white noise orthogonal to the past, and extend $\varphi (\nu ),\sigma (\nu )$ periodically in $\nu$. A nondegenerate causal periodic autoregressive model of order one requires $|{\prod }_{\nu =1}^T\varphi (\nu )|<1$. Write $V_{\nu }={\gamma }_{\nu }(0)$, with season indices understood modulo $T$.

For $h\ge 1$, multiply the recursion by $X_{nT+\nu -h}$ and use orthogonality of the current innovation to the past. The periodic Yule-Walker equations areAt lag zero, expansion of the squared recursion givesIn particular, ${\gamma }_{\nu }(1)=\varphi (\nu )V_{\nu -1}$, so an equivalent variance equation is $V_{\nu }=\varphi (\nu ){\gamma }_{\nu }(1)+\sigma (\nu {)}^2$. Negative lags are obtained from the given symmetry ${\gamma }_{\nu }(-h)={\gamma }_{\nu +h}(h)$.

Replacing covariances by their sample versions givesTake the nonnegative scale and require a positive estimated predecessor variance. Using matched seasonal sample covariance matrices makes the quantity under the square root nonnegative by Cauchy-Schwarz inequality; with unmatched covariance estimates one can instead fit seasonal ordinary least squares and use the nonnegative residual variance. Statistical consistency of the covariance estimates gives statistical consistency of these plug-in estimators. If the driving noise variance were not normalized to $1$, the equations would identify $\sigma (\nu {)}^2\mathrm{Var}({\epsilon }_t)$, not the two scales separately.

The Gauss-Markov theorem states that with full-column-rank design, mean-zero errors and covariance ${\sigma }^{2}I$, the ordinary least squares estimator is the best linear unbiased estimator: its covariance is no larger than any other such estimator's in the Loewner order. Normality is unnecessary for that theorem.

In the given normal linear model, maximizing the likelihood function is equivalent to minimizing $\Vert Y-X\beta {\Vert }^2$. The normal equations and full rank giveFor a direct variance comparison, write ${\beta }^{\ast }=AY$. Unbiasedness for every $\beta$ requires $AX=I_p$. Set $B=(X^{T}X{)}^{-1}{X}^T$ and $D=A-B$, so $DX=0$. The cross covariance terms vanish because $B{D}^T=(X^{T}X{)}^{-1}(DX{)}^T=0$. ThereforeThis proves the requested comparison for every contrast $t$ without relying only on the theorem's name.

Choose knots independently of the eventual validation responses, and let $b(x)=(b_1(x),\dots ,b_K(x){)}^T$ be a natural cubic spline basis as in part (a). A regression spline writes $m(x)=b(x{)}^T\beta$. Define the design matrix $B_{ij}=b_j(X_i)$. Under the stated normal linear model, maximum likelihood estimation for $\beta$ is the same as ordinary least squares:Differentiation gives the normal equations $B^{T}B\hat{\beta }=B^{T}Y$. If the design has full column rank,If it does not, a Moore-Penrose inverse gives the fitted values, but not every coefficient or off-sample prediction is identifiable. Merely choosing distinct knots does not guarantee full rank for an arbitrary set of observed predictor values.

More knots allow greater curvature and reduce approximation bias at the cost of larger estimation variance. Too many or poorly located knots can cause unstable fits. Equally spaced knots are simple; empirical predictor quantiles put more flexibility where observations are dense; subject knowledge can concentrate knots near likely changes. Both the number and positions can be compared using cross-validation or an appropriate penalized model-selection criterion. Adaptive knot selection must be included within validation, since treating a data-selected basis as prespecified understates its flexibility. Boundary knots should cover the region where inference is intended, and natural linear tails do not by themselves justify far-out extrapolation.

For $\lambda >0$, minimize the smoothing spline objective over the Sobolev space of functions with square-integrable second derivative on the observed interval:The roughness penalty measures curvature; its null space consists of affine functions. To see why the minimizer is a natural cubic spline, let $g$ be the natural spline interpolating the values of any competitor $m$ at its distinct observation locations, and put $h=m-g$. Integration by parts twice on each knot interval gives $\int g^{\prime \prime }{h}^{\prime \prime }=0$: $g^{\prime \prime \prime \prime }=0$ on each interval, $h$ vanishes at the knots, $g^{\prime \prime }$ is continuous, and $g^{\prime \prime }$ vanishes at the outer endpoints. Hencewhile both functions have the same residual sum of squares. This proves the reduction to a finite-dimensional spline space.

For distinct $X_i$ and $n\ge 2$, use its $n$-dimensional basis, write $B_{ij}=b_j(X_i)$, and define the positive semidefinite matrixThe penalized least squares problem is $\Vert Y-B\beta {\Vert }^2+\lambda {\beta }^T\Omega \beta$, soUse linear continuation outside the observation interval. The fitted-value matrix is $S_{\lambda }=B(B^{T}B+\lambda \Omega {)}^{-1}{B}^T$; its trace gives the effective degrees of freedom. As $\lambda$ decreases to zero, the natural spline approaches interpolation, with trace $n$; as it grows without bound, the fit approaches ordinary least squares on $1,x$, with trace two. For $\lambda =0$ the criterion alone does not determine values between observations; the natural interpolant is the limiting convention. Repeated observation locations are combined by using distinct knots and their multiplicities as weights, rather than pretending the spline design has $n$ independent interpolation values.

Ignoring the bounded second-order remainder, this is the linear regression modelThe ordinary least squares estimate through the known zero intercept isbecause $Z_i^2=1$. This is the randomized symmetric finite-difference derivative estimator.

Use the shape-rate convention for the Gamma distribution. DefineThe zero null space assumption makes $B$ a positive-definite matrix, and $u_{\ast }$ is the unique ordinary least squares solution. The Gaussian likelihood contributes ${\gamma }^{k/2}$, which must be retained because $\gamma$ is unknown. Multiplying it by the improper prior and the Gamma distribution gives the normal-gamma posterior with a flat prior:Here the constant density of the improper prior cancels on normalization. PutThe normal equation $A^{T}r=0$ makes the residual orthogonal to the range of $A$. The orthogonal decomposition by a closed subspace into that range and its orthogonal complement givesIntegration over $u$ by the Gaussian integral, followed by integration over $\gamma$ using the Gamma distribution, yieldsThus the posterior distribution is proper, despite the improper prior; here $k\ge d$ follows from the zero null space assumption.

Completing the square in $u$ and collecting the powers of $\gamma$ give the two conditional distributions:Explicitly their probability density functions areFor a joint maximum a posteriori estimate relative to $dud\gamma$, first fix $\gamma >0$. The unique maximizing $u$ is $u_{\ast }$. The remaining log density is $(a-1)\log \gamma -b_0\gamma$ plus a constant. Consequently, if $a>1$,The stated assumptions do not always imply $a>1$. If $a=1$, the supremum occurs as $\gamma \downarrow 0$ but is not attained on $\gamma >0$; if $a<1$, the density is unbounded there. In either case there is no joint maximum a posteriori estimate in the stated parameter space. Calling zero a boundary mode does not make it an admissible positive precision.

If “MAP estimators” instead means separate modes of the marginal distributions, integration givesThus the marginal maximum a posteriori estimate of $u$ is always $u_{\ast }$, and that of $\gamma$ is $(s-1)/b_0$ when $s>1$, with the same nonattainment at zero when $s\le 1$. Joint and marginal maximization are different operations; the usual joint interpretation gives the preceding boxed pair.

The Gaussian likelihood is proportional toso maximum likelihood estimation is equivalent to ordinary least squares. If $p\le n$ and $X$ has rank $p$, then $X^{T}X$ is invertible and the normal equations giveIf $p>n$, then $\mathrm{rank}X<p$ and $\ker X\ne \{0\}$. A least-squares minimizer always exists, but it is not unique: by rank-deficient ordinary least squares, the complete set isan affine subspace of dimension $p-\mathrm{rank}X>0$. Thus there are uncountably infinitely many likelihood maximizers. In the generic full-row-rank case their dimension is $p-n$.

Because $X$ has full column rank, $X^{T}X$ is invertible. The ordinary least squares estimator isIt satisfies the normal equationsso the residual is orthogonal to the column space of $X$. For any $\beta$,The two terms are orthogonal, and the Pythagorean theorem in an inner-product space givesThus $\hat{\beta }$ minimizes the least-squares objective.

For the normal linear model, the Gaussian likelihood givesMaximization over $\beta$ gives the ordinary least squares estimator $\hat{\beta }$, and maximization over the variance givesAt this value the quadratic term in $-2\ell$ is $n$. There are $p$ regression coefficients and one variance parameter, so substitution into the Akaike information criterion yields

The ordinary least squares equations for model 2 areFor the Gamma generalized linear model, $V(\mu )={\mu }^2$ and $d\mu /d\eta =\mu$, so its score equation under the logarithmic link isWhen $T_i$ is close to ${\mu }_i$,by the first-order Taylor expansion of the exponential function. The Gamma score equations then become the model-2 normal equations, so their coefficient estimates are close.