= Variational autoencoder
{wiki}
A \b[variational autoencoder] is a <generative model> with a <latent variable> $z$ and a learned approximation to its <posterior distribution> given an observation $x$. Write the model as $p_\theta(x,z)=p(z)p_\theta(x\mid z)$. A <recognition network>, implemented as a <neural network>, produces the parameters of $q_\phi(z\mid x)$; a <generative decoder> specifies $p_\theta(x\mid z)$. Using one recognition rule across observations is <amortized inference>, rather than solving a fresh independent optimization problem for every input.<Auto-Encoding Variational Bayes by Diederik P. Kingma and Max Welling>[§2.1–2.3, generative and recognition models; Introduction, neural-network recognition model]{ref}
Training maximizes an <evidence lower bound>, often written
$$
\mathcal L(\theta,\phi;x)=\mathbb E_{q_\phi(z\mid x)}[\log p_\theta(x\mid z)]-D_{\mathrm{KL}}(q_\phi(z\mid x)\Vert p(z)).
$$
The first term scores reconstruction under the declared likelihood; the second compares the inferred latent law with the prior. The identity
$$
\log p_\theta(x)=\mathcal L(\theta,\phi;x)+D_{\mathrm{KL}}(q_\phi(z\mid x)\Vert p_\theta(z\mid x))
$$
proves that it is a lower bound, since <Kullback-Leibler divergence> is nonnegative. It becomes tight when the recognition law equals the model posterior, under the usual density/support conditions.<Auto-Encoding Variational Bayes by Diederik P. Kingma and Max Welling>[§2.2, equations (1)–(3)]{ref}
For a one-dimensional Gaussian example, let the prior be $N(0,1)$ and the recognition law be $N(\mu,\sigma^2)$. Substituting their log densities and using $\mathbb E[z^2]=\mu^2+\sigma^2$ gives
$$
D_{\mathrm{KL}}=\frac12(\mu^2+\sigma^2-1-\log\sigma^2).
$$
At $\mu=1$, $\sigma=2$, this is $2-\log2\simeq1.30685$. The value is a penalty for this particular latent law, not the whole training loss. <Gaussian reparameterization> writes $z=\mu+\sigma\varepsilon$ with fixed-law standard-normal noise so that gradients can pass through the sampled latent value. After training, generation instead draws from the prior and then the <generative decoder>; it need not start from an observed input.
Back to article page
