= Normalizing flow
A \b[normalizing flow] constructs a <probability distribution> by passing a simple reference <random variable> through invertible differentiable transformations. The <change of variables formula> supplies its <probability density function>: if $Z=f(U)$ and $f$ is a <diffeomorphism> between equal-dimensional spaces, then
$$
q_Z(f(u))=q_U(u)\left|\det Df(u)\right|^{-1}.
$$
The <Jacobian determinant> accounts for local expansion or contraction of volume. Composing several transformations gives a flexible distribution while retaining a computable density, useful for a richer approximate <posterior distribution> in <variational inference>.<Variational Inference with Normalizing Flows by Danilo Jimenez Rezende and Shakir Mohamed>[§3, equations5–7, invertible transformations and composition]{ref}
For an original nonlinear example, let $U$ have the <standard normal distribution> with density $\varphi$, and set $f(u)=u+u^3$. Since $f'(u)=1+3u^2>0$ and $f(u)$ tends to opposite infinities at the two ends of the real line, every output has exactly one preimage. Its <probability density function> satisfies
$$
q_Z(u+u^3)=\frac{\varphi(u)}{1+3u^2}.
$$
At $u=1$, the output is $z=2$ and $q_Z(2)=\varphi(1)/4\simeq0.06049$. The factor $1/4$ reflects how strongly the transformation stretches nearby inputs. Integrating after the substitution $z=f(u)$ cancels this factor against $dz=f'(u)\,du$, leaving total probability one.
For $z_k=f_k(z_{k-1})$, the <chain rule> gives
$$
\log q_K(z_K)=\log q_0(z_0)-\sum_{k=1}^K\log\left|\det Df_k(z_{k-1})\right|.
$$
Generating transformed samples alone does not evaluate their <probability density function>; evaluating the determinant factors is essential. The single-preimage formula also requires invertibility: the map $u\mapsto u^2$ folds two real inputs together, so it cannot be substituted into this formula unchanged.
Back to article page
