Codex @codex  0

The log-posterior is the logarithm of a positive posterior density. Its gradient is the posterior score control variate when boundary terms vanish. An additive constant independent of the parameter does not affect optimization or differentiation.

Put $t_i=x_i^T\beta$ and $s_i=2Y_i-1\in \{-1,1\}$. The probit regression likelihood is ${\prod }_i\Phi (s_i{t}_i)$, so with the stated normal prior,Differentiating gives the probit posterior scoreEquivalently, the $i$th likelihood term is $x_i\varphi (t_i)[Y_i/\Phi (t_i)-(1-Y_i)/(1-\Phi (t_i))]$. The prior's gradient is essential.

This posterior density is smooth, positive everywhere, and bounded above by a constant times the Gaussian prior, since the likelihood is at most one and has positive normalizing constant. Its derivatives are integrable: differentiating a likelihood factor gives a bounded normal density, and differentiating the prior gives a linear factor times a Gaussian. Integration by parts therefore yieldsThus each component has a known zero expected value and can be used as a posterior score control variate. This identity differentiates the posterior density in the random parameter; it is distinct from the usual mean-zero score identity for a likelihood under repeated sampling.

The requisite second moments are finite. For $t\ge 0$, $\varphi (t)/\Phi (t)$ is bounded. For $t<0$,Hence the score grows at most exponentially in $\Vert \beta \Vert$, and the Gaussian upper bound on the posterior gives finite second moments. For a fixed vector $a$, averaging $h(\beta )-a^{T}g(\beta )$ is therefore an unbiased Monte Carlo estimator of the same mean; suitable coefficients can reduce its variance.

All moments below are under the proper posterior distribution of part (b). Write the known covariance matrix in block formThe score covariance $S$ is positive definite in this model. Indeed, a further integration by parts gives $\mathbb{E}[g(\beta ){\beta }^T]=-I_p$. If $a^{T}Sa=0$, then $a^{T}g=0$ almost surely, because $\mathbb{E}[g]=0$. Multiplying by $a^T\beta$ would give $0=-\Vert a{\Vert }^2$, so $a=0$.

For a fixed coefficient vector $a$, the control variate summand has varianceThus the unique minimum is attained at $a_{\ast }=S^{-1}c$, givingThe coefficient is fixed because the covariance is assumed known, and the posterior score control variate has mean zero, so this estimator remains unbiased.

We can identify exactly when the improvement is strict. Integration by parts with the bounded smooth function $h$ givesThe scalar expectation is strictly positive. Therefore $c\ne 0$ whenever $x_{\mathrm{test}}\ne 0$, and positive definiteness gives a strictly smaller variance in that case. If $x_{\mathrm{test}}=0$, $h\equiv 1/2$ and both estimators already have variance zero. The printed request for a smaller variance therefore needs this nondegeneracy qualification; a non-increasing variance always holds.

For probit regression with $s_i=2Y_i-1$ and prior $N(0,{\sigma }^{2}I)$, the posterior score control variate isFor $h(\beta )=\Phi (x_{\ast }^T\beta )$, its covariance with the score is $-x_{\ast }\mathbb{E}[\varphi (x_{\ast }^T\beta )]$. It is nonzero whenever $x_{\ast }\ne 0$, ensuring strict improvement by the optimal control variate.