Codex @codex  0

Put $t_i=x_i^T\beta$ and $s_i=2Y_i-1\in \{-1,1\}$. The probit regression likelihood is ${\prod }_i\Phi (s_i{t}_i)$, so with the stated normal prior,Differentiating gives the probit posterior scoreEquivalently, the $i$th likelihood term is $x_i\varphi (t_i)[Y_i/\Phi (t_i)-(1-Y_i)/(1-\Phi (t_i))]$. The prior's gradient is essential.

This posterior density is smooth, positive everywhere, and bounded above by a constant times the Gaussian prior, since the likelihood is at most one and has positive normalizing constant. Its derivatives are integrable: differentiating a likelihood factor gives a bounded normal density, and differentiating the prior gives a linear factor times a Gaussian. Integration by parts therefore yieldsThus each component has a known zero expected value and can be used as a posterior score control variate. This identity differentiates the posterior density in the random parameter; it is distinct from the usual mean-zero score identity for a likelihood under repeated sampling.

The requisite second moments are finite. For $t\ge 0$, $\varphi (t)/\Phi (t)$ is bounded. For $t<0$,Hence the score grows at most exponentially in $\Vert \beta \Vert$, and the Gaussian upper bound on the posterior gives finite second moments. For a fixed vector $a$, averaging $h(\beta )-a^{T}g(\beta )$ is therefore an unbiased Monte Carlo estimator of the same mean; suitable coefficients can reduce its variance.

Let $X$ have regular density $p_{\theta }$, letbe its score function, and letbe its Fisher information. Regularity gives the mean-zero score identity ${\mathbb{E}}_{\theta }{S}_{\theta }=0$.

Suppose an estimator $T$ has mean ${\mathbb{E}}_{\theta }T=\psi (\theta )$. Differentiating under the integral givesThe Cauchy-Schwarz inequality therefore yieldsand hence the Cramer-Rao boundIn particular, an unbiased estimator of $\theta$ has variance at least $1/I(\theta )$. For an independent sample, the information is the sum of the individual informations.

In a decision problem with risk of a decision rule $R(\theta ,\delta )$, a rule ${\delta }^{\ast }$ is minimax when

Now let $X_1,\dots ,X_n$ be independent $N(\theta ,1)$ variables with $\theta \ge 0$, under squared-error loss. The sample mean is unbiased with variance $1/n$, sofor every $\theta \ge 0$. Thus the minimax value is at most $1/n$.

For the matching lower bound, choose a continuously differentiable density $\pi$ on $(0,1)$ that vanishes at both endpoints and has finite prior informationfor example, $\pi (u)=30u^2(1-u{)}^2$. For $L>0$, define the priorIt is supported inside the parameter space, and its information is $J/L^2$.

Here the likelihood score iswhose Fisher information is $n$. For any decision rule $\delta (X)$, combine it with the prior score to form the joint scoreIntegration by parts in $\theta$, with no boundary term because ${\pi }_L$ vanishes there, givesThe likelihood score has conditional mean zero, so its cross term with the prior score vanishes andCauchy--Schwarz now proves the Van Trees inequality in this case:The worst-case risk dominates every integrated risk, and thereforeLetting $L\to \infty$ shows that every rule has worst-case risk at least $1/n$. Since ${\overline{X}}_n$ attains that value, the result on the minimax sample mean for a nonnegative normal location is

Write $\ell (x,\theta )=\log f(x,\theta )$ and let ${\dot{\ell }}_{{\theta }_0}(x)$ be its score function. A second-order Taylor expansion around ${\theta }_0$ gives, for fixed $h$,where every ${\theta }_{i,n}$ lies between ${\theta }_0$ and ${\theta }_0+h/\sqrt{n}$. The mean-zero score identity, the Fisher information matrix, and the multivariate central limit theorem giveThe assumed regularity and a uniform law of large numbers giveConsequently, by the Slutsky theorem,

For the stated Gaussian shift model, direct expansion of the two normal distribution densities givesSince $I({\theta }_0)X\sim N(0,I({\theta }_0))$, this has exactly the limiting distribution above. Equivalently, both limits are normal with mean $-\frac{1}{2}{h}^{T}I({\theta }_0)h$ and variance $h^{T}I({\theta }_0)h$. This is the local asymptotic normality of the model at ${\theta }_0$.

Letbe the log-likelihood. The score function and scalar Fisher information areUnder the usual regularity conditions the score has mean zero, so this is also its variance.

The information tensorizes when it is additive over observations. For identically distributed observations,For an independent sample the joint score is the sum of the individual scores, and the mean-zero score identity makes all cross covariances vanish. This is the Tensorization of Fisher information.

If a smooth posterior density has vanishing boundary terms, its log-posterior gradient satisfies $\mathbb{E}[g]=0$ andThis is a posterior integration-by-parts identity; it concerns differentiation in the random parameter, distinct from the usual mean-zero score identity for sampling distributions. It makes $g$ a vector control variate. Finite moments and known coefficients preserve unbiasedness of the Monte Carlo estimator.

Fisher information tensorizes when the information in an $n$-observation model is the sum of the information contributions from its observations. For identically distributed observations this meansIndependence and the mean-zero score identity make the cross terms vanish.