Normalizing flow

A normalizing flow constructs a probability distribution by passing a simple reference random variable through invertible differentiable transformations. The change of variables formula supplies its probability density function: if $Z=f(U)$ and $f$ is a diffeomorphism between equal-dimensional spaces, thenThe Jacobian determinant accounts for local expansion or contraction of volume. Composing several transformations gives a flexible distribution while retaining a computable density, useful for a richer approximate posterior distribution in variational inference.^[ref]

For an original nonlinear example, let $U$ have the standard normal distribution with density $\phi$, and set $f(u)=u+u^3$. Since $f^{\prime }(u)=1+3u^2>0$ and $f(u)$ tends to opposite infinities at the two ends of the real line, every output has exactly one preimage. Its probability density function satisfiesAt $u=1$, the output is $z=2$ and $q_Z(2)=\phi (1)/4\simeq 0.06049$. The factor $1/4$ reflects how strongly the transformation stretches nearby inputs. Integrating after the substitution $z=f(u)$ cancels this factor against $dz=f^{\prime }(u)du$, leaving total probability one.

For $z_k=f_k(z_{k-1})$, the chain rule givesGenerating transformed samples alone does not evaluate their probability density function; evaluating the determinant factors is essential. The single-preimage formula also requires invertibility: the map $u\mapsto u^2$ folds two real inputs together, so it cannot be substituted into this formula unchanged.