Past exam of the mathematics course of the University of Cambridge / 2003 / iii / Paper 41 / 6 / viii

A curved exponential family restricts a higher-dimensional full canonical family to a lower-dimensional smooth parameter surface:The derivative matrix $J(\theta )=\partial \eta /\partial \theta$ has rank $q$. A genuinely curved surface is not merely an affine subfamily under a choice of natural coordinates. With $n$ iid observations the ambient sufficient statistic is $T_n={\sum }_{i}t(X_i)$, and the parameter score and information areThe second-derivative terms from $\eta (\theta )$ have zero expectation because the centered sufficient statistic has mean zero. Under regularity, the constrained MLE is still first-order asymptotically normal with this information. Curvature affects higher-order bias, likelihood geometry and efficiency; concavity of the ambient likelihood need not survive along the nonlinear parameter surface.

For example, $N(\mu ,{\mu }^2)$ with $\mu >0$ has $t(x)=(x,x^2)$ and natural parameters ${\eta }_1=1/\mu$, ${\eta }_2=-1/(2{\mu }^2)$, constrained by ${\eta }_2=-{\eta }_1^2/2$. It has one free parameter but generally still needs both $\sum X_i$ and $\sum X_i^2$ as sufficient statistics. To see why, a likelihood ratio between two samples is constant in $\mu$ only if their statistic differences satisfy $\Delta T_1/\mu -\Delta T_2/(2{\mu }^2)=\text{constant}$ for all positive $\mu$, forcing both differences to zero. Lower parameter dimension therefore does not generally reduce the minimal sufficient statistic to that dimension.