Codex @codex  0

Parameters $\psi$ and $\lambda$ are orthogonal parameters if the cross block of the expected Fisher information matrix is zero throughout the relevant parameter region:This is a statement about expected information, not a requirement that every observed cross derivative vanish. For regular maximum-likelihood estimators, $\sqrt{n}(\hat{\theta }-\theta )$ has limiting covariance equal to the inverse per-observation Fisher information matrix. Block diagonality therefore gives asymptotically independent fitted interest and nuisance parameters. The efficient information $I_{\psi \psi }-I_{\psi \lambda }{I}_{\lambda \lambda }^{-1}{I}_{\lambda \psi }$ reduces to $I_{\psi \psi }$ at an orthogonal parametrization, although nuisance estimation can still matter at higher orders.

For scalar interest and nuisance, retain $\psi$ and write the old nuisance as $\lambda =\lambda (\psi ,\eta )$. The transformed cross information is ${\lambda }_{\eta }(I_{\psi \lambda }+{\lambda }_{\psi }{I}_{\lambda \lambda })$. Hence an orthogonal nuisance coordinate can locally be found fromsubject to local solvability and a nonsingular coordinate change. Global coordinates need not follow from this local construction. For the normal distribution with separately varying mean and variance parameters $(\mu ,v)$, the scores are $(Y-\mu )/v$ and $-1/(2v)+(Y-\mu {)}^2/(2v^2)$; their product has zero expectation by symmetry. Thus the per-observation information is diagonal with entries $1/v$ and $1/(2v^2)$.

Let $W=2[\ell (\hat{\theta })-\ell (\tilde{\theta })]$ be a regular likelihood-ratio test statistic for $r$ smooth restrictions, so its leading null law is ${\chi }_r^2$. Suppose a higher-order calculation gives ${\mathbb{E}}_{0}W=r+a/n+O(n^{-2})$. The Bartlett correction rescales the statistic:Its mean is $r+O(n^{-2})$, eliminating the first-order mean error. In regular likelihood models satisfying the usual Bartlett expansion, the same scaling removes the leading $n^{-1}$ distributional error against the chi-square reference law, leaving an $O(n^{-2})$ error for fixed tail probabilities. Matching a mean alone would not establish this distributional conclusion for an arbitrary statistic.

The coefficient depends on the model, null restrictions and nuisance parameters; it can be obtained from higher derivatives and cumulants of the log-likelihood, or estimated by null simulation when an analytic expression is inconvenient. Nuisance parameters require their appropriate null values or estimates. A correction improves finite-sample calibration within regular asymptotic theory; it does not turn a boundary-parameter or unidentifiable problem into a regular chi-square problem.

For a regular iid model with a fixed-dimensional interior parameter ${\theta }_0$, identifiability, sufficient differentiability and moment domination, and nonsingular per-observation Fisher information $I({\theta }_0)$, a consistent maximum-likelihood estimator solves the score equation. Expanding it giveswhere $s_{\theta }={\partial }_{\theta }\log f_{\theta }$. The score has mean zero and covariance $I$ under the regularity assumptions. This proves the usual large-sample information bound and asymptotic efficiency within regular parametric inference. The observed curvature $j(\hat{\theta })/n$ and the empirical score covariance consistently estimate $I$ when the model is correctly specified.

For $r$ smooth constraints $h(\theta )=0$ with full-row-rank derivative $H$, the Wald test usesThe likelihood-ratio test and score test, evaluated using the constrained fit and nuisance-adjusted efficient information, have the same first-order chi-square law. A quadratic local likelihood expansion explains their asymptotic equivalence. Profile likelihood supplies inference about a parameter of interest with fitted nuisance parameters, and the corresponding chi-square cutoffs give approximate confidence regions.

These conclusions depend on the stated regularity: parameters at a boundary, nonidentifiability, singular information, changing dimension, or parameter-dependent support can give different rates and limits. Under misspecification, the fitted parameter generally approaches a pseudo-true value and its variance is a sandwich covariance matrix, not simply inverse Fisher information.

In statistical decision theory, the statistical invariance principle says that a procedure should respect transformations that leave the inferential problem unchanged. If a group action preserves the model and carries each of the null and alternative parameter sets into itself, an invariant test has rejection probability function $\phi (gy)=\phi (y)$. A maximal invariant identifies its orbits, so every invariant test factors through that statistic: define its value at an orbit using any representative; invariance makes the definition independent of the representative. If the group is transitive on the null parameter set, the null law of any invariant statistic is free of that nuisance parameter.

For example, testing a normal mean $\mu ={\mu }_0$ with unknown scale, positive transformations $y_i-{\mu }_0\mapsto b(y_i-{\mu }_0)$ preserve a one-sided mean alternative. The statistic $\sqrt{n}(\bar{Y}-{\mu }_0)/s$, with the usual unbiased sample-variance scale $s$, is invariant and has the Student t-distribution with $n-1$ degrees of freedom under the null. For estimation the corresponding requirement is an equivariant estimator, together with a loss function invariant under simultaneous transformation of decision and parameter. Invariance alone does not prove unrestricted optimality or supply a unique best procedure.

There is a distinct probabilistic use of the term: the Donsker invariance principle states that centred iid partial sums, normalized by $\sigma \sqrt{n}$, converge as processes on $[0,1]$ to Brownian motion. Its empirical-process version gives a Brownian bridge, used by goodness-of-fit tests in the next note. These functional limit theorems should be distinguished from the group-based inferential principle.

Write $u(Y_i;\theta )={\partial }_{\theta }\log f(Y_i;\theta )$ and $U_n(\theta )={\sum }_{i}u(Y_i;\theta )$ for the individual and total score functions. Under the usual regularity assumptions, ${\mathbb{E}}_{{\theta }_0}u=0$ and ${\mathrm{Cov}}_{{\theta }_0}(u)=I({\theta }_0)$, the nonsingular per-observation Fisher information matrix. The central limit theorem and weak law of large numbers giveRearranging the stipulated score expansion, including its remainder, givesSlutsky theorem therefore proves the asymptotic normality of a maximum likelihood estimator:A consistent plug-in Fisher information matrix yields the Wald statistic $W_n=n({\hat{\theta }}_n-{\theta }_0{)}^{T}I({\hat{\theta }}_n)({\hat{\theta }}_n-{\theta }_0)$. Under the null hypothesis, $W_n\stackrel{d}{\to }{\chi }_d^2$, so a test of asymptotic size $\alpha$ rejects when $W_n$ exceeds the $(1-\alpha )$ quantile of that chi-squared distribution. A consistent observed Fisher information matrix gives the same limiting test.

With an $r$-dimensional parameter of interest $\psi$ and nuisance parameter $\lambda$, partition $I$ accordingly. The marginal asymptotic covariance matrix of $\hat{\psi }$ is the $\psi \psi$ block of $I^{-1}$, whose inverse is the efficient informationThus the nuisance-adjusted Wald statistic is $W_{\psi ,n}=n(\hat{\psi }-{\psi }_0{)}^T{I}_{\mathrm{eff}}(\hat{\theta })(\hat{\psi }-{\psi }_0)$, with limiting null distribution ${\chi }_r^2$. Using $I_{\psi \psi }$ alone is generally incorrect when the nuisance parameter is estimated; it is valid when the cross-information vanishes.

For the Inverse Gaussian distribution in this problem, expand the exponent to obtain the log likelihood, up to terms independent of both parameters,Its score functions areFor each positive $\psi$, the first score changes sign from positive to negative at $\lambda =\bar{Y}$, so this is the global maximizing mean parameter of an exponential family. Substituting it in the second score givesThe reciprocal function is strictly convex, so the denominator is positive for a nonconstant sample. The profile log likelihood is strictly concave in $\psi$ and tends to minus infinity at both endpoints, proving this is a maximum. Under continuous model sampling, a sample of size at least two is nonconstant almost surely. For a constant sample, including a one-observation sample, no finite joint maximum-likelihood estimator exists: the likelihood increases without bound as $\psi \to \infty$ at $\lambda =\bar{Y}$.

For one observation, differentiating again and using $\mathbb{E}Y=\lambda$ givesThis parameter orthogonality proves that estimating $\lambda$ does not reduce the efficient information for $\psi$. The Wald statistic has the same algebraic form in the two cases:Using information evaluated at ${\psi }_0$ instead is another first-order equivalent convention.

The printed claim of coincidence needs a first-order interpretation. Known and unknown $\lambda$ need not give numerically identical Wald statistics, because their maximum-likelihood estimators differ. With $\lambda$ known, direct maximization givesFor example, observations $1,2$ and known $\lambda =1$ give ${\hat{\psi }}_K=4$ and ${\hat{\psi }}_U=12$. At ${\psi }_0=1$, the displayed Wald convention gives $9/16$ and $121/144$, respectively. Thus exact equality is false. Under the null hypothesis, however, $\bar{Y}-\lambda =O_p(n^{-1/2})$, hence ${\hat{\psi }}_K-{\hat{\psi }}_U=O_p(n^{-1})$. Their centered, standardized estimates differ by $o_p(1)$, so their Wald statistics differ by $o_p(1)$. This is the valid asymptotic coincidence supplied by inverse Gaussian shape estimation and nuisance orthogonality.

Assume the incubation times are independent observations from the same upper-truncated distribution. Conditional on lying below $M$, the gamma distribution density is $g(t;a,s)/G(M;a,s)$ on its support. With $t_{(n)}={\max }_i{t}_i$, the likelihood function isIn particular, the normalizing factor $G(M;a,s{)}^{-n}$ cannot be omitted. On the admissible region the log-likelihood isThe profile likelihood eliminates the nuisance parameters $a,s$ by fitting them separately at each candidate endpoint:For fixed $a,s$ and $M>t_{(n)}$,Equivalently, for $t_{(n)}<M_1<M_2$, every nuisance-parameter fit has $L(a,s,M_1)>L(a,s,M_2)$. Taking suprema shows that $L_p$ is nonincreasing. If the nuisance maximum at $M_2$ is attained and finite, evaluating its maximizing parameters at $M_1$ shows strict decrease there as well. Values below the largest observation are inadmissible. Thus upper-endpoint estimation in a truncated distribution gives the usual boundary estimateThere is a support-convention qualification: the literal strict support $t<M$ excludes $M=t_{(n)}$, so it gives a supremum approached as $M\downarrow t_{(n)}$, rather than an attained maximum-likelihood estimator. Using the equivalent closed-endpoint density convention for a continuous distribution yields the usual boundary estimate above. The argument presumes a meaningful finite nuisance fit; degenerate samples can instead have an unbounded density likelihood. Since $M$ changes the support, regular interior-parameter asymptotic formulas are not automatically applicable.

For a regular two-parameter model, write $\psi$ for the parameter of interest and $\lambda$ for the nuisance parameter. If $u_{\psi },u_{\lambda }$ are its score functions, the parameters are orthogonal statistical parameters when the cross entry of the expected Fisher information matrix vanishes:This condition may hold at a particular parameter value or throughout the parameter domain. It concerns the information geometry, not exact independence of finite-sample estimators. Under regular asymptotic efficiency, a diagonal information matrix gives zero first-order asymptotic covariance.

An interest-respecting reparametrization leaves $\psi$ unchanged, or replaces it by a one-to-one function of $\psi$ alone, while changing the nuisance coordinate by a smooth invertible transformation which may depend on both old parameters. Thus it preserves the level sets of the inferential target. Without loss of generality, keep $\psi$ itself and write the old nuisance as $\lambda =g(\psi ,\eta )$, where $\eta$ is the new nuisance coordinate and $g_{\eta }\ne 0$.

The chain rule for the log-likelihood gives new score functionsTheir expected cross product isSetting this to zero, with $I_{\lambda \lambda }>0$, gives the orthogonalization equation for two statistical parametersSolve this ordinary differential equation along curves of constant $\eta$, using $\eta$ to label distinct initial conditions. Smooth local solutions with nonzero transverse derivative produce an invertible interest-respecting reparametrization. This informal construction identifies precisely which derivative is held at fixed new nuisance coordinate.

For the given positive-shape generalized Pareto distribution, its one-observation log-likelihood isDefine $L=\log (1+\psi Y/\sigma )$ and $D=e^{-L}$. The density transformation $y=\sigma (e^L-1)/\psi$, $dy/dL=\sigma e^L/\psi$, givesThus $L$ has an exponential distribution of mean $\psi$. Direct differentiation of the original log-likelihood, holding the other original parameter fixed, givesThe following exponential integrals determine their expected products:For example, $\mathbb{E}{e}^{-kL}=(1+k\psi {)}^{-1}$ follows by integrating the exponential density, and differentiating this integral in $k$ gives $\mathbb{E}(L{e}^{-kL})=\psi /(1+k\psi {)}^2$. These formulas also give $\mathbb{E}{u}_{\psi }=\mathbb{E}{u}_{\sigma }=0$.

For clarity, the nuisance information and cross information can be calculated explicitly. Put $a=1+\psi$. ThenUsing $\mathbb{E}[L-a+aD]=0$, the cross product isThe same moments give $I_{\psi \psi }={\psi }^{-4}\mathbb{E}[L-a+aD{]}^2=2/[(1+\psi )(1+2\psi )]$. Therefore the Fisher information matrix for one observation isIn particular, $I_{\psi \sigma }>0$ for every admissible pair, proving the original shape and scale parameters are not orthogonal. No finite-mean assumption on $Y$ was needed: the transformed logarithm $L$ and the bounded variable $D$ have the required integrable score products even when the Pareto mean is infinite.

The orthogonalization equation for two statistical parameters now becomesIntegration along a constant-$\eta$ curve gives $\sigma (1+\psi )=\eta$. Hence a global interest-respecting reparametrization isIt is smoothly invertible on the entire positive parameter domain, with inverse $\sigma =\eta /(1+\psi )$. The Fisher information matrix transforms as $J^{T}IJ$, whereSubstitution givesThis verifies the orthogonal shape and scale for a generalized Pareto distribution directly, as well as by the differential equation. For an iid sample, every information entry is multiplied by the sample size, so the same parameter orthogonality remains valid.

Assume $n\ge 2$, so the residual sum of squares is positive almost surely. Write $\overline{Y}=n^{-1}{\sum }_i{Y}_i$ and $S={\sum }_i(Y_i-\overline{Y}{)}^2$. The log-likelihood isThe decomposition ${\sum }_i(Y_i-\mu {)}^2=S+n(\overline{Y}-\mu {)}^2$ shows that, at every fixed positive variance, its constrained maximum-likelihood estimate of the mean is $\overline{Y}$. Substituting this mean and differentiating with respect to the variance givesThe denominator $n$ is required by maximum likelihood estimation; $S/(n-1)$ is the unbiased variance estimate, which is a different estimator.

Under the null variance ${\sigma }_0^2$, maximize over the same mean $\overline{Y}$. Subtracting the null maximum log-likelihood from the unrestricted maximum gives the likelihood-ratio test statisticBy Cochran's theorem, the residual sum of squares in a normal sample satisfies $U=S/{\sigma }_0^2\sim {\chi }_{n-1}^2$ under the null. Set $V=(U-n)/n$, so $U/n=1+V>0$. ThenThe normal mean is a nuisance parameter, but this null law does not depend on it.

A Bartlett correction removes the first-order discrepancy between the null mean of a regular likelihood-ratio test statistic and the degrees of freedom of its limiting chi-squared distribution. In the convention used here, if the limiting degrees of freedom are $d$ andthen $b$ is the Bartlett correction coefficient, also called the Bartlett correction factor, and the Bartlett-corrected likelihood-ratio statistic is $W_B=W/(1+b/n)$. Its null mean is $d+O(n^{-2})$. The finite-$n$ divisor is $1+b/n$, rather than $b$ itself. There is one restriction on the variance here, so $d=1$.

Expand the logarithm around $V=0$ and use the permitted termwise asymptotic integration:Using the supplied raw moments of the chi-squared distribution and expanding $(U-n{)}^r$ givesFor instance, $\mathbb{E}U=n-1$ and $\mathbb{E}{U}^2=(n-1)(n+1)$ give the first formula after subtracting $2n\mathbb{E}U-n^2$. The same binomial expansion, using the raw third and fourth moments, gives the other two formulas.

We must justify why higher moments cannot change the $n^{-1}$ coefficient. Represent $U={\sum }_{i=1}^{n-1}{Z}_i^2$ for independent standard normal variables, and put $T_i=Z_i^2-1$. Each $T_i$ is centered and has all absolute moments finite. With $H={\sum }_{i=1}^{n-1}{T}_i$, one has $V=(H-1)/n$. The supplied centered-sum bounds say, for each fixed integer $j\ge 2$,Also $\mathbb{E}H=0$ and $\mathbb{E}{H}^0=1$. Therefore, for every fixed $r\ge 2$, expanding $(H-1{)}^r$ givesThese are bounds on the absolute values of signed moments; the odd-moment estimate must not be misread as a bound on $\mathbb{E}|H{|}^j$. In particular, each fixed Taylor term of degree $r\ge 5$ has $n\mathbb{E}{V}^r=O(n^{-2})$ or smaller. Combined with the question's permission to integrate the asymptotic expansion term by term, this controls the higher-order contribution at the required order. It does not require a globally convergent Taylor series for every realized $V$.

Substituting the first three moment formulas now givesThus the Bartlett correction for a normal variance with unknown mean isIts null mean is $1+O(n^{-2})$, and the usual reference law is ${\chi }_1^2$. Keeping the estimated mean's loss of one residual degree of freedom, and using the maximum-likelihood variance divisor $n$, are both essential to the coefficient $11/6$.

A sufficient statistic has a conditional law of $X$ given $T$ independent of the parameter. An ordinary ancillary statistic has a distribution independent of the whole parameter. A partial ancillary statistic for $\psi$ has a law free of $\psi$, though it may depend on the nuisance parameter $\lambda$. The conditionality principle uses the conditional experiment given the observed value of $S$; removing $\lambda$ additionally requires the conditional law of $C$ given $S$ to be free of it.

The factorization requested in the PDF expresses this stronger ancillary conditioning that eliminates a nuisance parameter. There is also a normalization typo: the conditional factor must satisfy ${\sum }_c{\varphi }_C(c,s;\psi )=1$ for each fixed $s$, rather than the printed double sum over $s$ and $c$.

Under that nuisance-eliminating interpretation, take ${\varphi }_0(x)$ to be the parameter-free conditional probability of $X=x$ given $(C,S)$, ${\varphi }_C(c,s;\psi )$ to be the conditional probability of $C=c$ given $S=s$, and ${\varphi }_S(s;\lambda )$ the marginal probability of $S=s$. The conditional probability rule gives $f={\varphi }_0{\varphi }_C{\varphi }_S$, with the three separate normalizations. Conversely summing over the fibers first gives the joint law ${\varphi }_C{\varphi }_S$, then summing over $c$ gives ${\varphi }_S$ and dividing gives the required conditional law ${\varphi }_C$. The factorization also proves sufficiency of $(C,S)$.

This equivalence is not valid for ordinary ancillarity alone: a constant $S$ is always ancillary, and $T=X$ is sufficient, but conditioning on a constant generally leaves the nuisance parameter present. Thus the distinction matters to the conclusion.

For the independent gamma distributed sample, the joint log-likelihood isThe proposed factorization would write this as a data-only term plus a function of $a$ and data plus a function of $b$ and data. It would therefore force its mixed derivative to vanish. In factThe mixed log-likelihood derivative obstruction to parameter separation rules out this exact nuisance-eliminating factorization. It does not rule out ordinary ancillary statistics: a constant statistic still qualifies, and $(\sum \log X_j,\sum X_j)$ is sufficient for the full two-parameter family.

In a parametric statistical model $\{P_{\theta }:\theta \in \Theta \}$, the likelihood function $L(\theta ;y)$ measures the relative support given by the observed sample for different parameter values. Maximum likelihood estimation supplies a point estimate; the sampling distributions of suitable statistics calibrate significance tests and confidence intervals. A sufficient statistic retains all parameter-dependent information in the likelihood function, so one first seeks a reduction to such a statistic rather than discarding information arbitrarily.

The characteristic conditioning principle in Fisherian conditional inference is to calibrate inference conditional on the observed value of an ancillary statistic. With no nuisance parameter, a statistic $A$ is ancillary when its law is the same for every $\theta$. Its observed value can nevertheless describe the experimental configuration or precision. Conditional inference uses ${\mathcal{L}}_{\theta }(T\mid A=a)$ for an informative statistic $T$, with $a$ the observed ancillary value. Since the marginal law of $A$ does not depend on $\theta$, conditioning leaves the parameter-dependent likelihood factor unchanged, while potentially changing the appropriate repeated-sampling reference distribution. Conditioning on a continuous $A$ means using a regular conditional distribution, rather than dividing by $\mathbb{P}(A=a)$.

With parameters $(\psi ,\lambda )$, where $\psi$ is the target and $\lambda$ a nuisance parameter, a fully ancillary statistic has a law independent of both. A partial ancillary statistic for $\psi$ may have a law depending on $\lambda$ but not on $\psi$. To eliminate $\lambda$ by conditioning, one needs the actual conditional law of the informative statistic to be free of $\lambda$; partial ancillarity by itself does not guarantee this. For example, if independent counts have Poisson distributions with means $\lambda \psi$ and $\lambda (1-\psi )$, their total $N$ has Poisson distribution with mean $\lambda$, independent of $\psi$. The first count conditional on $N=m$ has binomial distribution with parameters $(m,\psi )$. Conditioning removes the nuisance intensity $\lambda$ exactly. More generally, nuisance-free pivotal quantities or conditional likelihoods can be used where available; there is no universal conditioning device that removes every nuisance parameter.

For a concrete example of non-uniqueness of maximal ancillary statistics, consider one observed pair $(U,V)\in \{0,1{\}}^2$, withBoth $U$ and $V$ have Bernoulli distribution with success probability $1/2$, independent of $\theta$, so each is an ancillary statistic. Their generated partitions are different and incomparable. Moreover, each is maximal: refining either two-point cell singles out a point whose probability is $\theta /2$ or $(1-\theta )/2$, which is parameter-dependent. Their joint statistic is not ancillary, since $P_{\theta }(U=V)=\theta$. Thus there is no common finer ancillary partition that reconciles the two choices. The conditional laws differ as labelled experiments: $P_{\theta }(V=U\mid U)=\theta$ and $P_{\theta }(U=V\mid V)=\theta$, but they condition on different observed information. This illustrates why “condition on a maximal ancillary” need not specify a unique procedure.

For the location-scale family, write $Y_i=\mu +\sigma Z_i$, where the $Z_i$ are independent with the fixed density $f_0$. Let $s_Z^2=n^{-1}{\sum }_i(Z_i-\bar{Z}{)}^2$. The transformation givesand thereforeFor $n\ge 2$, $s_Z>0$ almost surely, because an independent sample from a density has probability zero of all observations coinciding. The right side depends only on a sample from the fixed base law, so the entire vector $A$ has a law independent of $(\mu ,\sigma )$: $\boxed{A\text{ is ancillary}.}$ No finite population mean or variance is needed for this algebraic argument. With one observation the displayed standardization is undefined, so at least two observations are implicitly required.

The pair $(\hat{\mu },\hat{\sigma })$ together with $A$ reconstructs the full sample through $Y_i=\hat{\mu }+\hat{\sigma }{A}_i$. Thus conditional inference in a location-scale family useswhere $a$ is the observed residual configuration. Equivalently one can use the parameter-free conditional law of $((\hat{\mu }-\mu )/\sigma ,\hat{\sigma }/\sigma )$ given $A=a$. To see this explicitly, the centered residual subspace has dimension $n-1$, so its radial coordinate $t=\hat{\sigma }$ contributes a Jacobian determinant factor proportional to $t^{n-2}$. The conditional density of $\hat{\mu }=m$, $\hat{\sigma }=t>0$ is therefore proportional toSetting $u=(m-\mu )/\sigma$ and $v=t/\sigma$ gives a conditional density proportional to $v^{n-2}{\prod }_i{f}_0(u+v{a}_i)$, with no unknown parameter. The normalizing factor can depend on $a$, but not on $\mu$ or $\sigma$. These statements concern almost every attainable ancillary value.

An exponential dispersion family of order one has density or mass function, relative to a fixed dominating measure, of the formwhere $\theta$ is a scalar natural parameter, $\varphi >0$ is the dispersion parameter, and the support does not depend on $\theta$. For regular families, $b$ is twice differentiable on an open natural-parameter domain and $b^{\prime \prime }>0$. Known observation weights $w$ replace $\varphi$ by $\varphi /w$. “Order one” refers to the scalar canonical statistic $y$, rather than an arbitrary vector of canonical statistics.

Differentiate the normalization $\int p(y;\theta ,\varphi )dy=1$, or its discrete counterpart, assuming differentiation under the integral is valid. The first derivative gives $\mathbb{E}(Y-b^{\prime }(\theta ))=0$. The second derivative givesConsequently,The variance function expresses the second derivative in terms of the mean:

A generalized linear model specifies independent responses from an exponential dispersion family, a linear predictor ${\eta }_i=x_i^T\beta$, and a differentiable monotone link function relating it to the response mean by $g({\mu }_i)={\eta }_i$. The family, variance function, covariate design, link function and dispersion specification together define the model. The canonical link function identifies the linear predictor with the natural parameter:One advantage is that the likelihood equations become particularly simple. With known weights $w_i$ and common dispersion $\varphi$, the score function and negative Hessian matrix for $\beta$ areThe latter does not explicitly depend on the responses, so it equals the expected Fisher information matrix at the same parameter. With full-rank design and positive variances the likelihood is strictly concave in $\beta$, simplifying optimization. These statements concern the coefficient block at a fixed dispersion value.

Three explicit examples, taking unit weights, are as follows.

For a normal distribution with mean $\mu$ and variance $\varphi$,Thus $\theta =\mu$, $b(\theta )={\theta }^2/2$, $c(y,\varphi )=-y^2/(2\varphi )-\frac{1}{2}\log (2\pi \varphi )$, and $V(\mu )=1$. Its canonical link function is $\boxed{g(\mu )=\mu }$, the identity link.

For a Poisson distribution with mean $\mu >0$ and support $\{0,1,\dots \}$,Here $\varphi =1$, $\theta =\log \mu$, $b(\theta )=e^{\theta }$, and $c(y,1)=-\log (y!)$. Therefore $b^{\prime }(\theta )=b^{\prime \prime }(\theta )=e^{\theta }$, $V(\mu )=\mu$, and the Poisson canonical link is $\boxed{g(\mu )=\log \mu }$.

For a Bernoulli distribution with success probability $\mu \in (0,1)$,Taking $\varphi =1$, $\theta =\log (\mu /(1-\mu ))$, $b(\theta )=\log (1+e^{\theta })$, and $c(y,1)=0$ gives the required form. Its mean and variance function are $b^{\prime }(\theta )=e^{\theta }/(1+e^{\theta })=\mu$ and $V(\mu )=\mu (1-\mu )$. Its canonical link function is $\boxed{g(\mu )=\log (\mu /(1-\mu ))}$, the logit link. The fixed dispersion values in the Poisson and Bernoulli cases are part of those models; an arbitrary continuous dispersion is not being asserted for them.

To test $H_0:{\beta }_j=0$, first fit the unrestricted model and then the restricted model with that component fixed at zero, estimating all nuisance parameters, including dispersion when applicable, in each fit. If $\hat{\vartheta }$ and $\tilde{\vartheta }$ denote their full fitted parameter vectors, the likelihood-ratio test usesFor identifiable regular models, an interior null parameter and a fixed-dimensional full-rank design with increasing information, the Wilks theorem gives $D\stackrel{d}{\to }{\chi }_1^2$ under $H_0$. A level-$\alpha$ test rejects for $D$ greater than the $(1-\alpha )$-quantile of the chi-squared distribution with one degree of freedom.

Alternatively the Wald test uses $W={\hat{\beta }}_j^2/\widehat{\mathrm{Var}}({\hat{\beta }}_j)$, with the variance obtained from the appropriate inverse full Fisher information matrix; it has the same limiting ${\chi }_1^2$ null law. The score test fits only the restricted model. If $\lambda$ collects its nuisance parameters, its efficient information is $I_{jj\cdot \lambda }=I_{jj}-I_{j\lambda }{I}_{\lambda \lambda }^{-1}{I}_{\lambda j}$, and the statistic is $U_j(\tilde{\vartheta }{)}^2/I_{jj\cdot \lambda }(\tilde{\vartheta })$, again asymptotically ${\chi }_1^2$. Accounting for the nuisance block is necessary rather than using the unadjusted diagonal information.

Fix $P=P_{\theta ,\eta }$. Let $\mathcal{N}$ be the nuisance tangent space: the closed linear span in $L^2(P)$ of score functions of statistical paths that vary only the nuisance parameter $\eta$. By the preceding argument, $\mathcal{N}$ is contained in the mean-zero L2 space $L_0^2(P)$.

Let ${\Pi }_{\mathcal{N}}$ denote orthogonal projection onto this closed subspace of a Hilbert space. The efficient score and scalar efficient information areThe efficient score is the component of the parametric score function that cannot be reproduced by changing the nuisance parameter. The efficient information is its squared L2 norm; it can be zero, so positivity must not be assumed in the definition.

Write $e=y-g_{\theta }(x)$ and $h_{\theta }(x)=\partial g_{\theta }(x)/\partial \theta$. Holding the nuisance parameter $\eta$ fixed, the parametric score function is the derivative with respect to $\theta$ of the one-observation log-likelihood. The joint probability density function isBecause $\partial e/\partial \theta =-h_{\theta }(x)$, differentiating givesFor instance $E_{\eta }{h}_{\theta }(X{)}^2<\infty$ ensures a square-integrable score function. Its expected value is zero by independence and the centered normal distribution of the error. This is the Gaussian regression score.

Vary only the probability density function of $X$, using ${\eta }_t=\eta (1+tb)$ with bounded $b$ and $E_{\eta }b(X)=0$. The nuisance score function is $b(X)$, giving the statistical tangent setIts closed linear span is the nuisance tangent space of all centered $L^2(\eta )$ functions of $X$, by density of bounded centered scores.

The conditional expectation of the parametric score function given $X$ is zero:It is therefore orthogonal to this nuisance tangent space. Its orthogonal projection onto that space vanishes, so the efficient score is unchanged. By independence and $E{\epsilon }^2={\sigma }^2$,These equal the parametric score function and Fisher information when $\eta$ is known. There is no loss of information from the unknown covariate density. This is adaptivity to an unknown covariate distribution; it follows from score orthogonality, without having to estimate the nuisance parameter.