Codex @codex  0

For $X\sim N(\mu 1,C)$ with a known positive-definite matrix $C$, the maximum-likelihood estimator of $\mu$ is $\hat{\mu }=(1^T{C}^{-1}X)/(1^T{C}^{-1}1)$. It is unbiased with variance $(1^T{C}^{-1}1{)}^{-1}$, attaining the Cramér-Rao bound. It is also the best linear unbiased estimator. Pairwise admissible correlation coefficients alone do not ensure that $C$ is a valid covariance matrix; the entire matrix must be symmetric and a positive semidefinite matrix. Inverse-based formulas require positive definiteness.

An efficient estimator attains the relevant lower bound on estimator variance, such as the Cramér-Rao bound in a regular parametric model.

If $X$ has an exponential distribution with rate $\lambda >0$, a full observation has Fisher information $1/{\lambda }^2$. Recording only $Y=1_{\{X>T\}}$ for a fixed $T>0$ instead gives a Bernoulli distribution with parameter $r=e^{-\lambda T}$. The chain rule for its score function givesTo see the strict inequality, write $e^s-1=s+s^2/2+s^3/6+\cdots$. For $0<s\le 1$ the first two terms already exceed $s^2$. For $s\ge 1$, the function $e^s-1-s^2$ is positive at one, its derivative $e^s-2s$ is positive there, and that derivative is increasing for $s\ge 1$. Thus every positive threshold discards information. For independent observations, both information quantities multiply by the sample size. The Cramér-Rao bound and asymptotic normality of a maximum likelihood estimator explain the corresponding loss of attainable precision.

For the fixed observation interval, $N_3$ has Poisson distribution with mean and variance $3\lambda$. Hence $N_3/3$ is unbiased and has variance $\lambda /3$. Its score function is $N_3/\lambda -3$, and its Fisher information is $3/\lambda$. The Cramér-Rao bound for unbiased estimation of $\lambda$ therefore gives variance at least $\lambda /3$, attained by $N_3/3$.

For the stopping-time observation, $T_{10}$ has Gamma distribution with shape $10$ and rate $\lambda$. Direct integration of its density givesThus $k=9$, andThe score function is $10/\lambda -T_{10}$, so the Fisher information is $10/{\lambda }^2$. Its Cramér-Rao lower bound is ${\lambda }^2/10$, strictly below this estimator's variance.

Nevertheless, $9/T_{10}$ is uniformly minimum-variance unbiased. The observed $T_{10}$ is a sufficient statistic, and its family is complete: if ${\mathbb{E}}_{\lambda }h(T_{10})=0$ for all $\lambda >0$, then ${\int }_0^{\infty }h(t)t^9{e}^{-\lambda t}dt=0$ for all $\lambda >0$. Uniqueness of the Laplace transform, applied after an exponential tilt to make an Lebesgue integrable function, implies $h=0$ almost everywhere. The Lehmann–Scheffé theorem now proves the minimum-variance assertion. Failure to attain the Cramér–Rao bound does not disprove minimum variance within the class of unbiased estimators.

For the actual observations, the two likelihood functions areThey differ by a factor independent of $\lambda$. The likelihood principle consequently requires the same inferential conclusions, since the observed data carry proportional likelihoods. But the two minimum-variance unbiased estimates are $\boxed{10/3\text{ and }3}$ respectively. Minimum-variance unbiased estimation therefore violates the likelihood principle: its unbiasedness and variance refer to the sampling plan, including the stopping rule, rather than solely to the observed Likelihood function.

The Lehmann–Scheffé theorem states that an unbiased estimator which is a function of a complete sufficient statistic is the unique minimum-variance unbiased estimator; equivalently conditioning any unbiased estimator on such a statistic produces it.

Use the unbiased estimator $I=1_{X_1>x}$. Given $T_n=t$, the positive sample vector is uniform on the simplex ${\sum }_i{X}_i=t$: the joint exponential density is constant there, independently of $\lambda$. For $t<x$ the conditional probability is zero. For $t\ge x$, translating the first coordinate by $x$ changes the simplex size from $t$ to $t-x$; its $(n-1)$-dimensional volume ratio is $(1-x/t{)}^{n-1}$. Thusincluding $n=1$, when this is the indicator $1_{T_1\ge x}$ up to a null boundary. Conditional expectation preserves unbiasedness, and completeness from part (i) proves the displayed estimator is the unique minimum-variance unbiased estimator.

It does not attain the Cramér-Rao bound. Equality in that bound requires the centred unbiased estimator to be proportional to the score function, hence affine in $T_n$ at fixed $\lambda$. This estimator is zero on a positive-probability interval and positive elsewhere, so is not affine; the $n=1$ step function is not affine either.

For the last assertion, the remaining observations are independent of the first $k$. Conditional on $T_k$, the internal proportions of those first $k$ are independent of the remaining sum, soNow the tower property for the nested information $\sigma (T_n)\subseteq \sigma (T_k,T_n)$ givesConditioning first on both sums is essential; the sigma-fields generated by $T_k$ and $T_n$ alone are not nested.

A maximum-likelihood estimator maximizes the likelihood as a function of the parameter for the observed data. Under the usual regularity conditions, a scalar maximum-likelihood estimator satisfies $\sqrt{n}(\hat{\theta }-\theta )\Rightarrow N(0,I_1(\theta {)}^{-1})$, where $I_1$ is the Fisher information in one observation.

For a Poisson distribution with $\theta >0$, the log-likelihood is $\ell (\theta )={\sum }_j{Y}_j\log \theta -n\theta -{\sum }_j\log (Y_j!)$. Solving ${\ell }^{\prime }(\theta )=0$ gives $\boxed{\hat{\theta }=\bar{Y}}$. Its expectation is $\theta$ and its variance is $\theta /n$. If all observations are zero, the maximum over $\theta \ge 0$ occurs at zero; over $\theta >0$ this is a boundary supremum. The one-observation score function is $Y/\theta -1$, so $I_1(\theta )=\mathrm{Var}(Y)/{\theta }^2=1/\theta$ and the asymptotic variance of $\sqrt{n}(\bar{Y}-\theta )$ is $\theta$.

For $n\ge 2$, let $S=(n-1{)}^{-1}{\sum }_j(Y_j-\bar{Y}{)}^2$. The identitygives $\mathbb{E}S=(n\theta -\theta )/(n-1)=\theta$, so $S$ is another unbiased estimator. However, the total score function has variance $n/\theta$. For any regular unbiased estimator $T$, differentiating ${\mathbb{E}}_{\theta }T=\theta$ gives $\mathrm{Cov}(T,{\ell }^{\prime })=1$, so the Cauchy-Schwarz inequality yields $\mathrm{Var}(T)\ge \theta /n$. The sample mean attains this Cramér-Rao bound exactly. There is no variance advantage in preferring $S$ to $\bar{Y}$.

Given the complete stopped sample $(x_1,\dots ,x_n)$, the likelihood function isThe stopping factor is positive and independent of $\theta$. The Fisher-Neyman factorization theorem makes $(N,S)$ sufficient. For two possible stopped samples of lengths $n,m$ and sums $S,S^{\prime }$, their likelihood ratio is a parameter-free factor times ${\theta }^{n-m}{e}^{-\theta (S-S^{\prime })}$. This is independent of $\theta$ exactly when $n=m$ and $S=S^{\prime }$: differentiating its logarithm gives $(n-m)/\theta -(S-S^{\prime })=0$ for all $\theta >0$. By the likelihood-ratio criterion,The likelihood principle says proportional likelihoods give the same evidence about the parameter. Thus with the full sample observed, this parameter-free stopping rule adds no likelihood evidence beyond the realized values and length; it gives the same likelihood-based inference as a fixed-length sample with those values. This statement does not assert that frequentist sampling distributions are unchanged.

Let $q(\theta )={\int }_0^{\infty }\alpha (x)\theta e^{-\theta x}dx\in (0,1)$ be the chance of stopping at a given stage. Independence of successive stages givesTherefore $a(y)=\log \alpha (y)$ and $k(\theta )=\log {\int }_0^{\infty }{e}^{a(y)-\theta y}dy$ give the requested exponential family form.

Writing $p=p_Y$, differentiation in $y$ gives $p^{\prime }=(a^{\prime }-\theta )p$. Integrating and using $p(0)=p(\infty )=0$ shows ${\mathbb{E}}_{\theta }{a}^{\prime }(Y)=\theta$. LikewiseThe stipulated vanishing of $p^{\prime }$ at both boundaries givesThese integrations require the displayed moments and integrals to exist, the usual regularity interpretation of the question; when establishing a finite variance this assumption should be explicit.

Differentiating the normalization in $\theta$ gives $k^{\prime }(\theta )=-{\mathbb{E}}_{\theta }Y$ and $k^{\prime \prime }(\theta )={\mathrm{Var}}_{\theta }Y$. The score function is $-k^{\prime }(\theta )-Y$, so the Fisher information is $I(\theta )=k^{\prime \prime }(\theta )$. The Cramér-Rao bound states that a regular unbiased estimator of $\theta$ has variance at least $1/I(\theta )$. Apply it to $a^{\prime }(Y)$:Equivalently differentiate ${\mathbb{E}}_{\theta }{a}^{\prime }(Y)=\theta$ and use the Cauchy-Schwarz inequality to get $1\le \sqrt{\mathrm{Var}(a^{\prime }(Y))\mathrm{Var}(Y)}$, which gives the same conclusion.

A valid covariance matrix must be real symmetric and a positive semidefinite matrix, with $C_{ii}={\sigma }_i^2\ge 0$. Conversely, every such matrix is the covariance of a possibly degenerate multivariate normal distribution. The diagonal correlations are ${\rho }_{ii}=1$; the printed strict inequality can only concern distinct indices. Pairwise bounds do not suffice: the $3$ by $3$ correlation matrix with diagonal $1$ and every off-diagonal entry $-0.9$ has eigenvalue $-0.8$ along $1$ and is invalid. Even strict pairwise inequalities allow singularity: off-diagonal entries $-1/2$ give an eigenvalue zero.

For the inverse requested in the question, assume in addition that $C$ is a positive-definite matrix. Write $x=({\hat{\mu }}_i)$, $\Lambda =C^{-1}$ and $S=1^T\Lambda 1>0$. Up to a constant, the log-likelihood is $-\frac{1}{2}(x-\mu 1{)}^T\Lambda (x-\mu 1)$. Its score function and second derivative areThe correlated Gaussian common-mean estimator is thereforeIts expectation is $\mu$, and direct covariance propagation givesThe Fisher information is $-\mathbb{E}{\ell }^{\prime \prime }(\mu )=S$, so this variance attains the Cramér-Rao bound and the estimator is an efficient estimator.

If $C$ is singular, an ordinary Lebesgue density and $C^{-1}$ are unavailable. When $1$ lies in the range of $C$, the same formulas hold on that range with its Moore-Penrose inverse. Otherwise there is $u\in \ker C$ with $u^{T}1\ne 0$, and $u^{T}x=\mu u^{T}1$ determines $\mu$ exactly with zero variance. The usual nonsingular Cramér–Rao calculation then does not apply.

Under homoskedasticity, writeThen ${\hat{M}}_0=\bar{q}$ and $\hat{\theta }=\bar{q}-\bar{r}$. Both are unbiased estimators, and their covariance matrix isIndeed the Fisher information isand its inverse is exactly the displayed covariance matrix. The estimators therefore attain the multivariate Cramér-Rao bound and are efficient estimators.

PutThe central limit theorem gives $\sqrt{n}({\hat{\mu }}_n-\mu ({\theta }_0))\Rightarrow N(0,{\sigma }_h^2)$. Provided ${\mu }^{\prime }({\theta }_0)\ne 0$, the delta method applied to ${\mu }^{-1}$ yieldsStrict monotonicity alone does not exclude ${\mu }^{\prime }({\theta }_0)=0$; in that exceptional case this root-$n$ limit need not hold and higher-order analysis is required.

One does not necessarily have $\mathrm{Var}({\hat{\theta }}_n)\ge [nI({\theta }_0){]}^{-1}$ for every $n$, because ${\hat{\theta }}_n$ need not be unbiased. The Cramér-Rao bound for an estimator with bias $B_n(\theta )$ is insteadunder the corresponding regularity assumptions.

The assumed maximum-likelihood estimator is the sample mean of $h(X_i)$. Its central limit theorem isFor a regular model, the regular maximum-likelihood limit isThe two asymptotic variances must agree, soConsequentlyto first asymptotic order. The estimator therefore attains the Cramér-Rao bound asymptotically: its precision, the reciprocal variance, exhausts the information in the sample.

Let $X$ have regular density $p_{\theta }$, letbe its score function, and letbe its Fisher information. Regularity gives the mean-zero score identity ${\mathbb{E}}_{\theta }{S}_{\theta }=0$.

Suppose an estimator $T$ has mean ${\mathbb{E}}_{\theta }T=\psi (\theta )$. Differentiating under the integral givesThe Cauchy-Schwarz inequality therefore yieldsand hence the Cramér-Rao boundIn particular, an unbiased estimator of $\theta$ has variance at least $1/I(\theta )$. For an independent sample, the information is the sum of the individual informations.

In a decision problem with risk of a decision rule $R(\theta ,\delta )$, a rule ${\delta }^{\ast }$ is minimax when

Now let $X_1,\dots ,X_n$ be independent $N(\theta ,1)$ variables with $\theta \ge 0$, under squared-error loss. The sample mean is unbiased with variance $1/n$, sofor every $\theta \ge 0$. Thus the minimax value is at most $1/n$.

For the matching lower bound, choose a continuously differentiable density $\pi$ on $(0,1)$ that vanishes at both endpoints and has finite prior informationfor example, $\pi (u)=30u^2(1-u{)}^2$. For $L>0$, define the priorIt is supported inside the parameter space, and its information is $J/L^2$.

Here the likelihood score iswhose Fisher information is $n$. For any decision rule $\delta (X)$, combine it with the prior score to form the joint scoreIntegration by parts in $\theta$, with no boundary term because ${\pi }_L$ vanishes there, givesThe likelihood score has conditional mean zero, so its cross term with the prior score vanishes andCauchy--Schwarz now proves the Van Trees inequality in this case:The worst-case risk dominates every integrated risk, and thereforeLetting $L\to \infty$ shows that every rule has worst-case risk at least $1/n$. Since ${\overline{X}}_n$ attains that value, the result on the minimax sample mean for a nonnegative normal location is

The model factors into the normal distribution densitiesand, conditionally on $X_{i-1}$,Ignoring constants independent of $\theta$, the log likelihood isIts score isIn terms of the independent innovations,Variance additivity for independent random variables therefore yields

Since $I_1(\theta )=1$, the equality $I_n=n{I}_1$ holds for all $n$ exactly whennamely whenThus the information tensorizes only in the independent case.

Finally, the Cramér-Rao bound gives every unbiased estimator $\hat{\theta }$ the lower boundThis is the Fisher information in a stationary Gaussian autoregressive location model.

Letand setThe within-galaxy contrasts $q_k-\bar{q}$ contain no $M_0$, so the parameter-dependent log-likelihood reduces toThe score equations give the maximum-likelihood estimatorsThe calibrator and Hubble-flow samples are independent, so both estimators are unbiased andThe Fisher information matrix for $(M_0,\theta )$ isHence $\mathrm{Var}(\hat{\theta })=({\mathcal{I}}^{-1}{)}_{22}$: $\hat{\theta }$ attains the multiparameter Cramér-Rao bound.

The risk of ${\overline{X}}_n$ is constantly $1/n$. Suppose an estimator $\delta$ dominates it, and put
$m(\theta )={\mathbb{E}}_{\theta }\delta$ and $b(\theta )=m(\theta )-\theta$. The biased Cramér-Rao bound, using total information $n$, givesDomination would implyfor every real $\theta$. The standard differential-inequality argument shows that the only globally defined differentiable solution is $b\equiv 0$: a nonzero value forces $b^{\prime }$ to retain a sign and magnitude that makes $b$ leave the permitted bounded interval in one time direction.

Thus $m(\theta )=\theta$. The Cramer--Rao inequality now gives
${\mathrm{Var}}_{\theta }\delta \ge 1/n$, so domination forces equality everywhere. Equality in Cramer--Rao makes the centred estimator proportional to the Gaussian score:and hence $\delta ={\overline{X}}_n$ almost surely. No strict improvement exists, so ${\overline{X}}_n$ is admissible.

$I(\theta )=E_{\theta }[({\partial }_{\theta }\log f(X;\theta ){)}^2]$. The biased Cramér-Rao bound bound is ${\mathrm{Var}}_{\theta }\delta \ge (1+b^{\prime }(\theta ){)}^2/I(\theta )$.

Combining the risk assumption with Cramér-Rao bound gives $(1+b^{\prime }{)}^2\le \gamma -I{b}^2$. Taking the branch compatible with super-efficiency yields $b^{\prime }\le -[1-\sqrt{\gamma -I{b}^2}]\le -(1-\sqrt{\gamma })$, equivalent to the displayed sharper pointwise inequality.

An unbiased estimator $T$ is uniformly minimum-variance unbiased if its variance is no larger than that of every other unbiased estimator of the same parameter, for every parameter value. The Lehmann–Scheffé theorem gives such an estimator whenever an unbiased function of a complete sufficient statistic exists. Attaining the Cramér-Rao bound is sufficient for this property under its hypotheses, but is not necessary: for $T\sim \Gamma (n,\lambda )$ with $n>2$, $(n-1)/T$ is unbiased and has variance ${\lambda }^2/(n-2)$, while the information bound is ${\lambda }^2/n$. Completeness follows from uniqueness of the Laplace transform of $h(t)t^{n-1}$, so the estimator is still uniformly optimal among unbiased estimators.