Past exam of the mathematics course of the University of Cambridge / 2021 / ii / Paper 1 / 29J (Principles of Statistics)

Letbe the log-likelihood. The score function and scalar Fisher information areUnder the usual regularity conditions the score has mean zero, so this is also its variance.

The information tensorizes when it is additive over observations. For identically distributed observations,For an independent sample the joint score is the sum of the individual scores, and the mean-zero score identity makes all cross covariances vanish. This is the Tensorization of Fisher information.

Write $f_{\theta }(x)$ for the joint density and assume regularity permits differentiation under the integral. Sinceone differentiation givesDifferentiating the score itself,Taking expectation givesThus the information identity is

The model factors into the normal distribution densitiesand, conditionally on $X_{i-1}$,Ignoring constants independent of $\theta$, the log likelihood isIts score isIn terms of the independent innovations,Variance additivity for independent random variables therefore yields

Since $I_1(\theta )=1$, the equality $I_n=n{I}_1$ holds for all $n$ exactly whennamely whenThus the information tensorizes only in the independent case.

Finally, the Cramer-Rao bound gives every unbiased estimator $\hat{\theta }$ the lower boundThis is the Fisher information in a stationary Gaussian autoregressive location model.