 Gradient Estimation Using Stochastic Computation Graphs by John Schulman and others

Primary conference paper. Section 3.1 and Appendix A derive the general gradient identity; Section 4 treats baselines. The construction combines random-choice probabilities and differentiable deterministic paths in a stochastic computation graph.