= Amortized inference
\b[Amortized inference] learns one shared rule that maps observations to approximate <posterior distributions>, spreading the work of fitting that rule across many inputs. Instead of separately optimizing variational parameters for every observation $x$, a <recognition network> returns the parameters of $q_\phi(z\mid x)$. The shared parameters $\phi$ are learned across data; evaluating the rule for a new input is then a different operation from fitting it.<Stochastic Backpropagation and Approximate Inference in Deep Generative Models by Danilo Jimenez Rezende and others>[§4, recognition model and joint optimization]{ref}
A scalar Gaussian model gives an exact example. Let $z\sim N(0,1)$ and $x\mid z\sim N(z,1)$. Completing the square gives
$$
p(z\mid x)=N(x/2,1/2).
$$
The shared rule $\mu_\phi(x)=ax+b$, with $a=1/2$, $b=0$ and fixed variance $1/2$, represents the exact posterior for every real $x$. Evaluating $x/2$ replaces a separate optimization for each observation in this stipulated model. It does not imply that every nonlinear <latent-variable model> has an exact rule this simple.
Sharing can introduce an additional restriction even when the chosen distribution family is adequate. For observations $x=-2$ and $x=2$, the exact posterior means are $-1$ and $1$. The Gaussian family $N(m,1/2)$ can represent each exactly when $m$ is fitted separately. If the shared rule is constrained to output one constant mean for both inputs, its best equal-weight choice is $m=0$. Each case then has
$$
D_{\mathrm{KL}}(N(0,1/2)\Vert N(\pm1,1/2))=1.
$$
This original example isolates an <amortization gap>: the loss from restricting how observation-specific variational parameters are produced, rather than an inability of the per-observation Gaussian family to express the target. The shared mapping's capacity and its training therefore matter alongside the family of <probability distributions>.
Back to article page
