Generative model
= Generative model
A \b[generative model] describes how observations can be produced. It can specify a <probability distribution> $p_\theta(x)$ explicitly, or provide a procedure that generates samples whose distribution is determined by parameters $\theta$. Producing observations and assigning a tractable numerical probability to every observation are separate capabilities.
An original two-bit example defines a <joint probability distribution> through <conditional probabilities>. Let
$$
\Pr(X_1=1)=0.8,\qquad
\Pr(X_2=1\mid X_1=1)=0.9,\qquad
\Pr(X_2=1\mid X_1=0)=0.2.
$$
Generate $X_1$ first, then generate $X_2$ using the applicable conditional probability. The four possible outputs have probabilities
$$
\Pr(1,1)=0.72,\quad\Pr(1,0)=0.08,\quad
\Pr(0,1)=0.04,\quad\Pr(0,0)=0.16,
$$
which sum to one. The procedure generates both complete observations and a dependence between their components. Always outputting the most probable observation, `11`, would produce a different distribution: it would discard all the other outcomes rather than sample the stipulated model.
A <generative adversarial network> gives a different implementation. A generator transforms an input random variable into a candidate observation, while a discriminator supplies a training signal by distinguishing generated samples from training examples. The original paper develops this as an adversarial learning procedure.<Generative Adversarial Nets by Ian Goodfellow and others>[Abstract and §2, generator/discriminator framework and sample generation]{ref}
The model's samples must be assessed against the intended distribution or task. A visually plausible observation alone does not establish that the model represents the full range of observations well; distributional coverage and the probability of different outcomes matter alongside individual sample appearance.