 Combined score and pathwise gradient estimator

Under fixed-support regularity permitting differentiation under the integral sign, apply the product rule to $\int s_{\lambda }{W}_{\lambda }$. The first derivative term is $\int W_{\lambda }\nabla s_{\lambda }=\mathbb{E}[W_{\lambda }\nabla \log s_{\lambda }]$ and the second is $\mathbb{E}[\nabla W_{\lambda }]$. This proves the displayed identity. For an evidence lower bound, take $W_{\lambda }=\log p(t_{\lambda }(\epsilon ),x)-\log q_{\lambda }(t_{\lambda }(\epsilon ))$, holding noise $\epsilon$ fixed in the inner derivatives. Parameter-independent continuous noise contributes a reparameterization gradient; untransformed discrete choices contribute a score-function gradient estimator. A mixed base law cannot be called parameter independent if its discrete factors retain variational parameters.^[ref]