Orthogonal statistical parameters ($I_{\psi \eta }=\mathbb{E}(u_{\psi }{u}_{\eta })=0$)

Two statistical parameters are orthogonal when their cross term in the expected Fisher information matrix vanishes. This is pointwise orthogonality at a specified parameter value, or global orthogonality when it holds throughout the parameter domain. It gives a zero cross covariance in the regular first-order asymptotic distribution of efficient estimators, rather than asserting exact finite-sample independence.

In a regular common-support statistical model with probability density function $a(\lambda ,y)e^{\lambda t(y;\psi )}$, the score function in $\psi$ is $\lambda t_{\psi }$. Its zero expectation gives $\mathbb{E}{t}_{\psi }=0$ for $\lambda \ne 0$. The mixed log-likelihood derivative is $t_{\psi }$, so the cross Fisher information matrix block vanishes. At zero the same conclusion follows by continuity when the regular derivatives extend there. This proves parameter orthogonality; it does not establish exact independence of fitted parameters.

In a normal linear model whose predictor columns are orthogonal to the intercept, two coefficient blocks are orthogonal statistical parameters precisely when the cross-block Fisher information is zero, equivalently the displayed design cross-product vanishes. Then $X^{\mathsf{T}}X$ is block diagonal, so the corresponding least-squares estimators have zero covariance and, being jointly Gaussian, are independent. Their fitted subspaces are orthogonal and their extra sums of squares do not depend on which block is added first.

For a regular interest/nuisance parameterization with parameter orthogonality, this nuisance-information adjustment approximates a conditional likelihood construction and reduces first-order nuisance effects. It is related to modified profile likelihood but is not a universal substitute for its sample-coordinate Jacobian. When the constrained and full nuisance fits coincide in the relevant ancillary coordinates, that Jacobian is one and the two expressions agree. For normal variance with unknown mean, it replaces the profile's $n$ by the residual $n-1$ in the logarithmic variance term.

For scalar interest parameter $\psi$, choose new nuisance coordinates $\eta$ and keep $\psi$ fixed. The transformed interest score function is $U_{\psi }+U_{\lambda }^{\top }{\partial }_{\psi }\lambda$, so the displayed differential equation makes its Fisher information cross-block with the nuisance score zero. Under smoothness and nonsingularity it has a local solution from initial coordinates on a fixed interest slice. Orthogonality is local expected-information decoupling; it implies first-order asymptotic independence of fitted blocks, not exact independence or nuisance-free conditional inference.

For the negative binomial distribution with mean $\mu >0$ and size $\theta >0$, the one-observation score function in the mean is $u_{\mu }=\theta (y-\mu )/[\mu (\mu +\theta )]$. Its size derivative is $(y-\mu )/(\mu +\theta {)}^2$, whose expectation is zero. Thus the expected Fisher information matrix is diagonal in $(\mu ,\theta )$. Regular interior maximum-likelihood estimators have zero asymptotic cross covariance, without necessarily being independent in finite samples.

An interest-respecting reparametrization preserves the parameter of interest, or changes it only by a one-to-one function of itself. The nuisance parameter may depend on both old parameters, provided the complete transformation is smoothly invertible. Thus the level sets representing the inferential target are unchanged. Orthogonality can be sought by choosing the new nuisance coordinate along solutions of the orthogonalization equation for two statistical parameters.

Write the old nuisance parameter as $\lambda =g(\psi ,\eta )$ with $g_{\eta }\ne 0$. The transformed score functions are $u_{\psi }^{\mathrm{new}}=u_{\psi }+g_{\psi }{u}_{\lambda }$ and $u_{\eta }=g_{\eta }{u}_{\lambda }$. Their expected cross product is $g_{\eta }(I_{\psi \lambda }+g_{\psi }{I}_{\lambda \lambda })$. Setting it to zero gives the displayed ordinary differential equation along fixed-$\eta$ curves, assuming positive nuisance Fisher information. Smooth solutions with distinct initial conditions give a local interest-respecting orthogonal coordinate system.