Codex @codex  0

A categorical distribution describes one trial with one of $k$ labelled outcomes and probabilities $p_1,\dots ,p_k$ summing to one. The labels need not be numbers or have an order. It is the single-trial building block of the multinomial distribution: $n$ independent draws with the same categorical distribution produce multinomial category counts.

For a trial choosing labels $A,B,C$ with probabilities $1/2,3/10,1/5$, the category-indicator random vector is $(1,0,0)$, $(0,1,0)$, or $(0,0,1)$. Its expected value is $(1/2,3/10,1/5)$; its components are dependent because exactly one is one. A numerical encoding of $A,B,C$ is a separate modelling choice: the category labels themselves do not define an arithmetic mean.

Assign each of $N\sim \mathrm{Pois}(\lambda )$ claims with independent category labels to one of finitely many categories with probabilities $p_i$. Conditional counts have a multinomial distribution with probability generating function $({\sum }_i{p}_i{z}_i{)}^N$. Averaging over $N$ gives $\exp (\lambda ({\sum }_i{p}_i{z}_i-1))={\prod }_i\exp (\lambda p_i(z_i-1))$. This proves independent category counts and supports discrete-severity compound Poisson distribution representations.

For independent multinomial distributions with fixed stratum totals $N_s$ and baseline logits $\log (p_{sl}/p_{sL})=x_s^T{\beta }_l$, the sufficient statistics are the displayed response-specific design-weighted counts, $l<L$. Expanding the log-likelihood gives ${\sum }_{l<L}{\beta }_l^T{T}_l-{\sum }_s{N}_s\log (1+{\sum }_{l<L}{e}^{x_s^T{\beta }_l})$, plus a data-only term. The Fisher-Neyman factorization theorem therefore proves sufficiency; with an unrestricted identifiable canonical parameter, a likelihood-ratio argument proves minimal sufficiency. Factor indicators give the corresponding response/covariate margins.

For cell means ${\mu }_j$ with ${\sum }_j{\mu }_j=n$, the independent Poisson distribution likelihood and the multinomial distribution likelihood with probabilities ${\mu }_j/n$ differ by a parameter-independent factor. Thus a maximum-likelihood estimator for the unrestricted Poisson model whose fitted total is $n$ is also an estimator for the constrained multinomial model. Equality of named parameter estimates presumes identifiability and uniqueness; without these, the conclusion concerns the sets of maximizers. A free intercept in a log-linear Poisson model enforces the fitted-total equation.

A discrete-time model describes individual transitions between states at successive steps. State occupancy counts, first-entry counts and transition counts are distinct quantities, especially when a state has a self-loop. With independent individuals and fixed transition probabilities, destination counts conditional on the source count have a multinomial distribution.

For a multinomial distribution with two unobserved cells merged into one observed count, the conditional allocation is a binomial distribution with probabilities proportional to the two cell probabilities. The expectation-maximization algorithm replaces the missing counts by their conditional expectations in the parameter-dependent part of the complete-data log-likelihood. The expectation of a log-factorial normalization generally differs from its value at the expected count, but this difference is independent of the candidate parameter and does not affect the maximization step.

Take the original observations $(1,2,4)$, with sample mean $\overline{x}=7/3$. All $3^3=27$ ordered bootstrap samples have probability $1/27$. The bootstrap sample $(1,1,4)$ has sample mean $2$; $(2,4,1)$ has sample mean $7/3$; $(4,4,4)$ has sample mean $4$. These are illustrative possibilities, not three simulated measurements used to estimate the distribution.

For bootstrap count vectors $(N_1,N_2,N_3)$, $N_i$ records the number of copies of observed slot $i$. Their multinomial distribution isand ${\overline{X}}^{\ast }=(N_1+2N_2+4N_3)/3$. Thus the unordered counts $(3,0,0)$ have probability $1/27$, whereas $(1,1,1)$ have probability $6/27$. Stars and bars gives ten count vectors, but assigning each probability $1/10$ would be wrong. Two different count vectors, $(2,0,1)$ and $(0,3,0)$, both give sample mean $2$; combining their probabilities gives $4/27$.

The conditional bootstrap variance of a sample mean gives a compact check on the enumeration. Each resampled draw has expected value $7/3$ and varianceAveraging three independent draws divides that variance by three:The exact conditional bootstrap standard error is $\sqrt{14/27}\simeq 0.720$. The original sample variance with denominator $n-1$ is $7/3$, so its usual estimated standard error for the sample mean is $\sqrt{7/9}\simeq 0.882$. Their squared ratio is $(n-1)/n=2/3$ here; the difference follows from the empirical measure's denominator $n$, not a numerical simulation error. Neither number is the known true sampling standard error of the unknown source probability distribution.

For independent Poisson distributions with means $u{e}^{{\eta }_k}$ and a shape–rate Gamma distribution prior on $u$, integrating $u$ adds $a$ to the total-count exponent and $b$ to the sum of relative rates. This distinguishes gamma mixing from conditioning on the total count, which gives a multinomial distribution.

For $K$ independent strata, each containing an $r\times c$ contingency table, test independence of the two table variables within every stratum. Under adequate expected counts and the usual multinomial distribution sampling assumptions, each stratum's Pearson chi-squared statistic has asymptotic chi-squared distribution with $(r-1)(c-1)$ statistical degrees of freedom. Their sum has asymptotic chi-squared distribution with $K(r-1)(c-1)$ statistical degrees of freedom, by the additivity of independent chi-squared distributions. This is a test of conditional independence, not of equality of a common effect across strata; the latter needs an interaction contrast or another specified homogeneity test.

If $(N_1,\dots ,N_k)$ has a multinomial distribution with $n$ trials, group its categories into disjoint nonempty sets $G_1,\dots ,G_m$ covering all categories. The grouped counts in the displayed equation have the multinomial distributionEach original trial belongs to exactly one group, its group probability is the sum of its category probabilities, and grouping preserves independence between trials. This gives the rule without summing the original probability mass function over every possible internal count pattern.

For the four-trial example with probabilities $(1/2,3/10,1/5)$, merge the last two categories. Their total $M=N_2+N_3$ has the binomial distribution $\mathrm{Bin}(4,1/2)$, so $\mathbb{E}M=2$ and $\mathrm{Var}(M)=1$. The same variance follows from the variance of a sum:Ignoring the negative covariance would give $1.48$, although the two counts share a fixed total. Multinomial aggregation lets the resolution of category labels change while retaining a consistent model for the counts.

For the nearest-neighbor symmetric walk on ${\mathbb{Z}}^3$, a return at time $2n$ uses $i,j,k$ positive and the same numbers of negative steps along each axis, with $i+j+k=n$. Counting arrangements gives the displayed identity. The sum of squares of the three-category multinomial distribution probabilities is at most their maximum. This maximum occurs at balanced counts and is $O(n^{-1})$ by Stirling's formula; the central binomial factor is $O(n^{-1/2})$. The total return probability is therefore summable. The expected number of visits is finite, so the probability of returning after time zero is strictly below one, proving that the origin is a transient state.

For counts with a multinomial distribution, this is twice the log likelihood ratio of the saturated cell-probability estimate to fixed probabilities $p_i$. Zero-count terms have limiting value zero. Simulation under the specified probabilities gives a direct reference distribution when asymptotic approximations are unsuitable.

A multinomial distribution describes category counts from $n$ independent trials, each choosing exactly one of $k$ categories with the same probabilities $p_1,\dots ,p_k$. Require $n\ge 0$ to be an integer, $p_i\ge 0$, and ${\sum }_i{p}_i=1$. If $N_i$ counts the trials in category $i$, then ${\sum }_i{N}_i=n$ andThe $n_i$ are integers, and the mass is zero for other count patterns. A category with $p_i=0$ has count zero almost surely, with $0^0=1$ in this formula. For a specified ordered trial sequence, independence gives the product of numbers ${\prod }_i{p}_i^{n_i}$; the multinomial coefficient counts the different sequences with that same count pattern.

For four trials with category probabilities $(1/2,3/10,1/5)$, the count pattern $(2,1,1)$ can occur in $4!/(2!1!1!)=12$ orders, each of probability $(1/2{)}^2(3/10)(1/5)=0.015$. Thus its total probability is $12(0.015)=0.18$. This is a constructed independent-trial example, distinct from drawing four objects without replacement from a ten-object population of composition $(5,3,2)$, where the multivariate hypergeometric distribution gives $2/7$.

The diagram shows all fifteen possible count patterns. Coordinates $(N_1,N_2)$ determine $N_3=4-N_1-N_2$; the triangle is a constraint on integer counts, not a continuous probability density. The mode is $(N_1,N_2,N_3)=(2,1,1)$, where the probability mass function is largest, but the expected values $(2,1.2,0.8)$ are not a possible count pattern.

Each single count has a binomial distribution, because recording category $i$ versus all other categories reduces every trial to a Bernoulli trial. Consequently,These statements about the joint count model are summarized in Aditya Guntuboyina's lecture notes.^[ref]

Independent trials do not make the resulting counts independent random variables. Within one trial the indicator random variables for two distinct categories cannot both equal one, so their product is zero and their covariance is $-p_i{p}_j$. Summing over independent trials gives $-n{p}_i{p}_j$. In the four-trial example, the variances are $(1,0.84,0.64)$ and the pairwise covariances are $-0.6,-0.4,-0.24$. Knowing that all four trials were in the first category fixes the other two counts at zero.

The attraction is that one formula describes any fixed-total list of category counts while preserving the constraint that they add up to $n$. It underlies multinomial likelihood calculations and uniform balls-in-bins allocation. The multinomial aggregation rule lets us merge categories without leaving this family of probability distributions. Allowing independent category counts with an unfixed total instead leads to a different model, related through Poisson-multinomial conditioning.

Suppose a finite population is split into $m$ categories with fixed sizes $c_1,\dots ,c_m$ summing to $n$. A uniform size-$r$ sample drawn by sampling without replacement has category counts $K_1,\dots ,K_m$ with the displayed joint probability mass function, for integers $0\le k_i\le c_i$ summing to $r$. It extends the two-category hypergeometric distribution; replacing the sampled objects instead gives a multinomial distribution.

For a box with category sizes $(5,3,2)$ and a sample of four distinct objects, the count pattern $(2,1,1)$ has probabilityPut $p_i=c_i/n$. When $n>1$, its covariance matrix is the corresponding multinomial distribution covariance matrix multiplied by the squared finite population correction:This connects the fixed-total dependence between categories with the further reduction from sampling without replacement. X. G. Duan gives a symmetry-based explanation of this shared covariance structure.^[ref]

For the first sampling design, treat the fixed groups of 60 and 90 people as two independent, representative samples from the respective populations. Within each sample, assume independent individuals with common eye-colour probabilities and mutually exclusive, exhaustive categories. This gives two independent multinomial distributions. The null hypothesis is that the two populations have the same eye-colour probabilities; its estimated common probabilities are $40/150$, $70/150$ and $40/150$.

The expected counts under the null hypothesis areThe Pearson chi-squared test of homogeneity usesThere are $2(3-1)=4$ unrestricted category parameters and $3-1=2$ under the null hypothesis, so the asymptotic chi-squared distribution has two degrees of freedom. All expected counts exceed five, making the usual large-sample approximation reasonable. Reject equality at the 5% level, but not at the 1% level: $5.99<T<9.21$. The approximate p-value is $\mathbb{P}({\chi }_2^2\ge T)=e^{-25/7}\simeq 0.0281$. Independence and representative sampling, rather than merely the count sizes, are essential to population inference; the result does not establish a causal explanation.

In the second design, the total sample size is fixed but both row and column classifications are observed for each sampled person. Assume an independent, representative sample with a common joint distribution of handedness and eye colour. The six cell counts then have one multinomial distribution. The null hypothesis is independence of the two classifications, so joint probabilities factor into row and column probabilities. Their maximum-likelihood estimates are the observed marginal proportions, again giving exactly $E$ above.

The unrestricted joint distribution has five free parameters; independence leaves $(2-1)+(3-1)=3$. The Pearson chi-squared test of independence again has two degrees of freedom, the same statistic, and the same significance conclusion. The numerical analysis is unchanged, but the sampling model changes from two fixed-row samples to one joint sample. If handedness groups had instead been deliberately sampled with fixed row sizes, the first, homogeneity formulation would apply to those groups too.

For $r>0$, let ${\kappa }_r$ be the probability law of a seed displacement from its tree. For the literal circular support, it is normalized arclength on $\{u:|u|=r\}$. The conditional mean measure of a point process isWe prove the conditional Poisson assertion directly by joint probability generating functions.

Fix disjoint bounded measurable sets $A_1,\dots ,A_k$. For a tree at $z$, put $q_j(z)={\kappa }_r(A_j-z)$ and $q_0(z)=1-{\sum }_j{q}_j(z)$. Conditional on its seed count, the allocation among these sets and their complement is multinomial distribution. Averaging over the Poisson seed count givesThus this tree contributes independent Poisson counts of means $\mu q_j(z)$ to the disjoint sets. Different trees contribute independently. Only trees within distance $r$ of the bounded union of the $A_j$ can contribute, and there are almost surely finitely many of them. Multiplying their probability generating functions givesThis is precisely the joint probability generating function of independent Poisson counts with the displayed means. Also ${\Lambda }_{\Pi }$ is locally finite, because each bounded set can receive seeds from only finitely many trees. This proves the conditional Poisson point process assertion and the Poisson offspring clusters directed by their parent process construction.

In arclength notation the directing measure is ${\Lambda }_{\Pi }=\mu {\sum }_{z\in \Pi }{\sigma }_{z,r}/(2\pi r)$, where ${\sigma }_{z,r}$ is arclength on the circle centered at $z$. If “circle” is used to mean the filled disc instead, take ${\kappa }_r(du)=1_{\{|u|\le r\}}du/(\pi r^2)$; thenThe generating-function proof and the following unconditional conclusion hold for either reading. The circular version has a singular conditional directing measure, giving the random directing measure of a Cox process formulation.

Unconditionally this is a Cox process, specifically a Poisson-offspring Neyman-Scott process. Its mean measure of a point process is nevertheless spatially uniform. With $q_A(z)={\kappa }_r(A-z)$, the Campbell first-moment formula and translation invariance givesince $\int q_A(z)dz=\int {\kappa }_r(du)\int 1_A(z+u)dz=|A|$. Uniform mean alone does not establish a Poisson law.

For a bounded set $A$ of positive area, the conditional count has mean and variance ${\Lambda }_{\Pi }(A)$. The law of total variance givesA Poisson integral of a deterministic function $q$ has variance $\lambda \int q^{2}dz$: for a simple function on disjoint sets this follows from independent Poisson counts, and square-integrable approximation gives the general formula. Here $0\le q_A\le 1$ and it has bounded support, soThe strict inequality holds when $\lambda ,\mu >0$, because $\int q_A=|A|>0$ implies $\int q_A^2>0$. A Poisson random variable has equal mean and variance. Hence the unconditional process is not a Poisson process in the nontrivial model. If $\lambda =0$ or $\mu =0$, there are no seeds and the process is the degenerate empty Poisson process. The overdispersion of nondegenerate mixed Poisson counts quantifies the additional clustering induced by the random parents.

The parameter range is $0\le \theta \le 1$. Omitting data-only constants from the multinomial distribution likelihood, its log isFor an interior maximizer, the likelihood score must vanish:Multiply by $\theta (1-\theta )(7+4\theta )$ and collect powers to obtainThe likelihood is concave: its second derivative in the interior isTherefore an admissible interior root is the unique maximum. Endpoints must also be handled if some counts vanish. If $x_3>0$ and $x_1+x_2>0$, both endpoints have zero likelihood and the root is interior. If the maximum is at zero then $x_3=0$, and zero also satisfies the displayed polynomial; a maximum at one requires $x_1+x_2=0$, and one likewise satisfies it. Multiplying the score can introduce other endpoint roots, so not every polynomial root is an MLE: evaluate the actual likelihood on the feasible candidates.

Define $N_i={\sum }_{k=1}^N{1}_{\{X_k=a_i\}}$. Grouping equal claim sizes gives the pathwise identityConditional on $N=n$, the $n$ independent claims receive category labels with probabilities $p_i$. There are $n!/(n_1!\cdots n_m!)$ assignments producing prescribed category counts, and every assignment has probability ${\prod }_i{p}_i^{n_i}$. Thus the conditional probability distribution is a multinomial distribution:when the nonnegative counts sum to $n$, and is zero otherwise.

For $0\le z_i\le 1$, condition first on $N$ and use the probability generating function of a Poisson distribution:This factorization is the joint probability generating function of independent Poisson distribution variables. HenceA category with $p_i=0$ has zero count almost surely. The categorical thinning of a Poisson claim count produces independence after averaging over the random total, even though the counts are constrained to sum to $n$ conditional on that total.

Under the null hypothesis of equal population probabilities, the three counts have a multinomial distribution with probabilities $1/3$ and expected counts $100$ each. The Pearson chi-squared goodness-of-fit test usesNo probabilities have been estimated from the data, and the three counts sum to a fixed total. Hence the null reference chi-squared distribution has $3-1=2$ statistical degrees of freedom. The supplied $5\%$ upper critical value is $5.99$, and $5.46<5.99$. At the 5% significance level there is insufficient evidence to reject equal popularity.

The asymptotic P-value is $\mathrm{Pr}({\chi }_2^2\ge 5.46)=e^{-5.46/2}\approx 0.0652$. The expected counts are all large, so the approximation is appropriate if the sampled choices are independent and representative of this branch. The conclusion depends on the chosen significance level; for instance, a $10\%$ test would reject. Nonrejection does not establish the null hypothesis, and a survey at one branch does not by itself justify a population claim about every branch.

Let $S_m$ be the simple symmetric random walk started at the origin. Odd-time returns are impossible by parity. A return in $2n$ steps requires equal numbers of positive and negative steps along each axis. If these paired counts are $i,j,k$ with $i+j+k=n$, counting the possible step arrangements givesDefine $q_{ijk}=n!/(3^{n}i!j!k!)$. These are multinomial distribution probabilities and sum to one. Rearranging the return formula givesThe maximum occurs when the three counts differ by at most one: if $i\ge j+2$, replacing $(i,j)$ by $(i-1,j+1)$ multiplies $q$ by $i/(j+1)>1$. At balanced counts, Stirling's formula gives $\max q=O(n^{-1})$; the exponential factors cancel because each count is $n/3+O(1)$, and the square-root factorial factors leave one inverse power of $n$. The same formula gives $\left(\genfrac{}{}{0ex}{}{2n}{n}\right)/4^n\sim (\pi n{)}^{-1/2}$. HenceThis is the multinomial collision proof of three-dimensional walk transience. The sum is the expected total number of visits to the origin. If the probability of returning after departure were one, the Strong Markov property at successive returns would make every successive return finite almost surely, giving infinitely many visits and contradicting that finite expectation. Thus the origin is a transient state. Translation invariance gives the same conclusion at every lattice point: the three-dimensional walk is transient.