Profile likelihood

For a parameter of interest $\psi$ and nuisance parameter $\lambda$, the profile likelihood isIt compares values of $\psi$ after fitting the nuisance parameter as favourably as possible at each value.

For parameter of interest $\psi$, constrained nuisance fit ${\hat{\lambda }}_{\psi }$, and suitable ancillary sample coordinates $(\hat{\psi },\hat{\lambda },a)$, this adjustment includes both nuisance observed information and a sample-space Jacobian determinant. The derivative holds $\hat{\psi },a$ fixed. Implicit differentiation of the constrained nuisance score function gives the equivalent expression ${\ell }_p+\frac{1}{2}\log |j_{\lambda \lambda }|-\log |{\ell }_{\lambda ;\hat{\lambda }}|$. The semicolon distinguishes data-coordinate differentiation from parameter differentiation. Omitting the Jacobian requires an additional justified simplification; the information adjustment alone is not universal.

For independent Inverse Gaussian distributions with shape $\psi$ and mean $\lambda$, the constrained and full nuisance maximum-likelihood estimates both equal $\bar{Y}$, giving a unit sample-coordinate Jacobian determinant. The nuisance observed information there is $n\psi /{\bar{Y}}^3$. Hence subtracting half its logarithm from the profile log-likelihood replaces the coefficient $n/2$ of $\log \psi$ by $(n-1)/2$. For a nonconstant sample the modified fit is $(n-1)/({\sum }_i{Y}_i^{-1}-n/\bar{Y})$.

For independent exponential distributions with means $\lambda e^{\psi x_i}$ and a nonconstant fixed design, put $A(\psi )={\sum }_i{Y}_i{e}^{-\psi x_i}$. The constrained maximum-likelihood estimate is ${\tilde{\lambda }}_{\psi }=A(\psi )/n$ and the profile log-likelihood is ${\ell }_p(\psi )=-n\log [A(\psi )/n]-\psi {\sum }_i{x}_i-n$. Use the ancillary statistic $a_i=\log Y_i-\log \hat{\lambda }-\hat{\psi }{x}_i$ to specify sample coordinates. At fixed $\hat{\psi },a$, the Jacobian determinant $\partial {\tilde{\lambda }}_{\psi }/\partial \hat{\lambda }={\tilde{\lambda }}_{\psi }/\hat{\lambda }$, while $j_{\lambda \lambda }(\psi ,{\tilde{\lambda }}_{\psi })=n/{\tilde{\lambda }}_{\psi }^2$. Therefore the modified profile likelihood satisfiesThe adjustment is independent of $\psi$. Keeping only the observed information adjustment would miss this cancellation.

For independent $X\sim \mathrm{Gamma}(m,\psi \lambda )$ and $Y\sim \mathrm{Gamma}(n,\lambda )$ in shape-rate notation, $T=X/Y$ has density proportional to ${\psi }^m{t}^{m-1}(1+\psi t{)}^{-(m+n)}$. Its distribution is nuisance-free in lambda. The nuisance maximum is $(m+n)/(\psi X+Y)$, so substituting into the full likelihood gives the same interest-parameter log-likelihood up to a parameter-free term. Equality here is special to the common-scale structure, not a general equality of profiling and marginalization.

The logarithm of the profile likelihood is the full log-likelihood evaluated at the nuisance fit constrained to the specified parameter of interest. It preserves likelihood ratios and maximizers, but is not generally a normalized sampling log-density. Conditioning and integrating out nuisance parameters are different operations.