Multinomial distribution (source code)

= Multinomial distribution
{title2=$\operatorname{Mult}(n;p_1,\ldots,p_k)$}
{wiki}

= Multinomial
{synonym}

A <multinomial distribution> describes category counts from $n$ <independent> trials, each choosing exactly one of $k$ categories with the same <probabilities> $p_1,\ldots,p_k$. Require $n\geq0$ to be an <integer>, $p_i\geq0$, and $\sum_i p_i=1$. If $N_i$ counts the trials in category $i$, then $\sum_iN_i=n$ and
$$
\mathbb P(N_1=n_1,\ldots,N_k=n_k)
=\frac{n!}{n_1!\cdots n_k!}\prod_{i=1}^k p_i^{n_i},
\qquad n_i\geq0,\quad\sum_i n_i=n.
$$
The $n_i$ are <integers>, and the mass is zero for other count patterns. A category with $p_i=0$ has count zero <almost surely>, with $0^0=1$ in this formula. For a specified ordered trial <sequence>, <independence> gives the <product of numbers> $\prod_i p_i^{n_i}$; the <multinomial coefficient> counts the different <sequences> with that same count pattern.

For four trials with category <probabilities> $(1/2,3/10,1/5)$, the count pattern $(2,1,1)$ can occur in $4!/(2!1!1!)=12$ orders, each of <probability> $(1/2)^2(3/10)(1/5)=0.015$. Thus its total <probability> is $12(0.015)=0.18$. This is a constructed independent-trial example, distinct from drawing four objects without replacement from a ten-object population of composition $(5,3,2)$, where the <multivariate hypergeometric distribution> gives $2/7$.

\Image[/multinomial-count-probabilities.png]
{title=Three category counts from four independent trials: every point fixes the third count through the total}

The diagram shows all fifteen possible count patterns. <Coordinates> $(N_1,N_2)$ determine $N_3=4-N_1-N_2$; the triangle is a constraint on <integer> counts, not a continuous <probability density>. The <mode> is $(N_1,N_2,N_3)=(2,1,1)$, where the <probability mass function> is largest, but the <expected values> $(2,1.2,0.8)$ are not a possible count pattern.

Each single count has a <binomial distribution>, because recording category $i$ versus all other categories reduces every trial to a <Bernoulli trial>. Consequently,
$$
\mathbb E N_i=np_i,\qquad
\operatorname{Var}(N_i)=np_i(1-p_i),\qquad
\operatorname{Cov}(N_i,N_j)=-np_ip_j\quad(i\ne j).
$$
These statements about the joint count model are summarized in Aditya Guntuboyina's lecture notes.https://stat238.berkeley.edu/spring-2026/lecturetwelve238spring2026/[Guntuboyina, Berkeley STAT 238, lecture 12: multinomial distribution]{ref}

Independent trials do not make the resulting counts <independent random variables>. Within one trial the <indicator random variables> for two distinct categories cannot both equal one, so their product is zero and their <covariance> is $-p_ip_j$. Summing over independent trials gives $-np_ip_j$. In the four-trial example, the <variances> are $(1,0.84,0.64)$ and the pairwise <covariances> are $-0.6,-0.4,-0.24$. Knowing that all four trials were in the first category fixes the other two counts at zero.

The attraction is that one formula describes any fixed-total list of category counts while preserving the constraint that they add up to $n$. It underlies <multinomial likelihood> calculations and <uniform balls-in-bins allocation>. The <multinomial aggregation> rule lets us merge categories without leaving this family of <probability distributions>. Allowing independent category counts with an unfixed total instead leads to a different model, related through <Poisson-multinomial conditioning>.