 Generative decoder

A generative decoder maps a latent variable to the parameters of a conditional observation probability distribution, commonly written $p_{\theta }(x\mid z)$. In a variational autoencoder, a neural network can supply those parameters. The conditional law, including its observation noise and support, is part of the model; the decoder need not return one deterministic reconstructed observation.^[ref]

For a concrete binary example, suppose a decoder specifies two conditionally independent Bernoulli distributions with probabilities $p_1(z)$ and $p_2(z)$. ThenIf a chosen latent value gives $(p_1,p_2)=(0.8,0.3)$, the conditional probability of $x=(1,0)$ is $0.8(1-0.3)=0.56$, and its negative log likelihood is approximately $0.5798$. The vector $(0.8,0.3)$ is a vector of probabilities, not itself a binary sample. A model for continuous observations would need a different conditional distribution.^[ref]

Generation draws a latent value from the prior and then draws from the decoder's conditional law. Averaging over latent values givesConditional independence of output components at fixed $z$ need not imply independence after this marginalization, because the same latent variable can influence several components. The recognition network runs in the other direction for inference; it is not an algebraic inverse of the generative decoder.