Multinomial distribution ($\mathrm{Mult}(n;p_1,\dots ,p_k)$)

The multinomial distribution gives the category counts from $n$ independent trials with category probabilities $p_1,\dots ,p_k$.

The vector of category counts is the sum of independent categorical indicator vectors. Their means are $p$ and covariance is $\mathrm{diag}p-p{p}^T$, so the multivariate central limit theorem gives the displayed limit. At uniform probabilities on $k$ categories, Pearson-standardized deviations have covariance equal to the orthogonal projection onto the $(k-1)$-dimensional subspace of zero-sum vectors. Their squared norm has an asymptotic chi-squared distribution with $k-1$ degrees of freedom.

Place $n$ labelled balls independently into $m$ labelled bins, each with probability $1/m$. The vector of bin counts has a multinomial distribution, and each count separately has the binomial distribution $\mathrm{Bin}(n,1/m)$. Distinct bin counts have covariance $-n/m^2$, so they are dependent for $n>0$ and $m>1$. Counts of bins of a specified size are sums of indicator variables; their expectations follow from the linearity of expectation, without any independence between bins.

In a uniform balls-in-bins allocation with $m\to \infty$ and $n/m\to \lambda \in (0,\infty )$, a fixed bin count has convergence in distribution to a Poisson distribution variable of mean $\lambda$ by the Poisson limit theorem. If $N_j$ counts bins containing exactly $j$ balls, symmetry and the linearity of expectation give $\mathbb{E}[N_j]/m=\mathbb{P}(B_1=j)\to e^{-\lambda }{\lambda }^j/j!$ for each fixed $j$. In particular the expected empty fraction tends to $e^{-\lambda }$, and the expected fraction with at least two balls tends to $1-(1+\lambda )e^{-\lambda }$. These are statements about expectations; convergence of the random fractions requires an additional argument.

For category counts $y_1,\dots ,y_k$ with fixed total $n$ and probabilities $p_1,\dots ,p_k$, the multinomial likelihood isIts maximum-likelihood fitted probabilities are ${\hat{p}}_j=y_j/n$.

For counts with a multinomial distribution, this is twice the log likelihood ratio of the saturated cell-probability estimate to fixed probabilities $p_i$. Zero-count terms have limiting value zero. Simulation under the specified probabilities gives a direct reference distribution when asymptotic approximations are unsuitable.

The categorical distribution describes one draw from finitely many categories with probabilities $p_1,\dots ,p_k$.