 Constant score baseline

For a normalized differentiable sampling law on fixed support, ${\mathbb{E}}_q[\nabla \log q]=\nabla \int q=0$. Consequently any deterministic scalar $b$, independent of the sampled value, can be subtracted from the reward in a score-function gradient estimator without changing its expectation. This is an identity for the estimator, not a licence to differentiate an added parameter-dependent objective without its other terms. For a scalar parameter and score $S$ with $\mathbb{E}[S^2]>0$, minimizing the estimator's variance gives $b^{\ast }=\mathbb{E}[h{S}^2]/\mathbb{E}[S^2]$, provided these moments exist: expand $\mathbb{E}[(h-b{)}^2{S}^2]$ and differentiate the quadratic in $b$. An arbitrary baseline need not reduce variance.^[ref]