Probit regression

Binary probit regression sets $p_i=\Phi ({\beta }_0+x_i^T\beta )$, where $\Phi$ is the standard normal distribution function. Its intercept score is a weighted sum of residuals, so an intercept does not generally force ${\sum }_i{\hat{p}}_i={\sum }_i{y}_i$.

For binary observations with success probability $\Phi (\beta x_i)$, introduce latent $Z_i\sim N(\beta x_i,1)$ and let the sign determine the observation. Given $\beta$ and the observed sign, each latent variable has a truncated normal distribution. With a standard normal prior, the full conditional distribution of $\beta$ is normal with precision $1+{\sum }_i{x}_i^2$ and mean ${\sum }_i{x}_i{Z}_i/(1+{\sum }_i{x}_i^2)$. Alternating these blocks is Gibbs sampling with the required marginal posterior.

Observing only whether a lognormally modelled output exceeds a known threshold produces a probit link. The latent location and slope are recovered by multiplying the probit coefficients by the known Gaussian scale and adding the log threshold to the intercept.

For probit regression with $s_i=2Y_i-1$ and prior $N(0,{\sigma }^{2}I)$, the posterior score control variate isFor $h(\beta )=\Phi (x_{\ast }^T\beta )$, its covariance with the score is $-x_{\ast }\mathbb{E}[\varphi (x_{\ast }^T\beta )]$. It is nonzero whenever $x_{\ast }\ne 0$, ensuring strict improvement by the optimal control variate.