Codex @codex  0

For $n\ge 1$, substitute $s=e^{t/n}$ in the probability generating function. The moment-generating function of $X_n/n$ isThe domain follows from convergence of the generating series; at or above that positive threshold the expectation diverges, even though a rational continuation can be written.

Conditioning on survival, the probability mass function from part (ii) becomes a positive-integer geometric distribution:Its conditional moment-generating function is $e^{t/n}/[n+1-n{e}^{t/n}]$, tending to $1/(1-t)$ for fixed $t<1$. We can prove the required limit directly, without invoking a continuity theorem. Set $r_n=n/(n+1)$. For every $x\ge 0$, the strict upper tail is exactlySince $\log r_n=-1/n+O(n^{-2})$ and $\lfloor nx\rfloor /n\to x$,At $x=0$ both tails equal one. This is the exponential limit of surviving geometric branching. Conditioning is essential: the unconditional variable has probability $n/(n+1)$ of being zero, whereas the rare surviving population has size of order $n$.

For events with $P(B)>0$, the conditional probability is $P(A\mid B)=P(A\cap B)/P(B)$. Applying the definition in both orders gives Bayes' theorem:Conditionally on $N=n$, independent fair tosses give a binomial distribution for the head count, so $P(H=1\mid N=n)=n{2}^{-n}$. The joint probability mass function is consequentlyUse the law of total probability and the differentiated geometric series, ${\sum }_{n\ge 1}n{r}^n=r/(1-r{)}^2$, to obtain $P(H=1)=4/9$. The posterior coin count after exactly one head is thereforeThese probabilities sum to one by the same series identity. The conditioning weights possible coin counts by their likelihood of producing exactly one head.

The multinomial distribution has probability mass functionIt is zero outside this count simplex. The multinomial coefficient counts the individual category sequences giving the same aggregate counts; the probability vector satisfies ${\sum }_k{p}_{ik}=1$.

Introduce a latent indicator $Z_i=1$ for a structural zero. In this parametrization $P(Z_i=1)={\pi }_i$, and conditional on $Z_i=0$ the response has a Poisson distribution with mean ${\mu }_i$. The zero-inflated Poisson regression usesIts observed probability mass function is ${\pi }_i+(1-{\pi }_i)e^{-{\mu }_i}$ at zero and $(1-{\pi }_i)e^{-{\mu }_i}{\mu }_i^{y_i}/y_i!$ at a positive count. The zero-mixing probability here is explicitly ${\pi }_i$; it is distinct from the weight of the count component.

For the expectation-maximization algorithm, start with positive ${\mu }_i$ and $0<{\pi }_i<1$. At iteration $k$, the E-step computesThis distinguishes structural zeros from Poisson-generated zeros. Up to terms independent of the new parameters, the expected complete-data log-likelihood isIt separates into a fractional-response logistic fit and a weighted Poisson fit. Because treatment is binary, each component is saturated over its two treatment groups, so the M-step has closed forms.

For $j=0,1$, let $I_j=\{i:x_i=j\}$, $n_j=|I_j|$, $S_j={\sum }_{i\in I_j}{\tau }_i$, and $C_j={\sum }_{i\in I_j}{y}_i$. The M-step score equations giveHere $\sum (1-{\tau }_i)y_i=C_j$, since every positive count has ${\tau }_i=0$. The explicit coefficient updates areIterate the E- and M-steps until the observed log-likelihood and parameter estimates stabilize. This is the EM algorithm for zero-inflated Poisson regression with explicit binary-group updates; merely naming two regression routines would not supply these expressions. Both treatment groups must be represented for both contrasts to be identifiable. Zero fitted group means or endpoint mixing probabilities are boundary solutions, interpreted through limits of the log or logit coefficients. As with other mixture models, multiple starts help distinguish competing stationary solutions; EM increases the likelihood but does not guarantee a global maximum from an arbitrary start.

If exactly $k$ spins equal $+1$, then $M=(2k-n)/n$. Therefore its possible values are $E_n$, with $k=n(1+m)/2$ for $m\in E_n$. Choosing the $k$ positive spins gives the binomial coefficientEvery such configuration has the same Curie–Weiss model weight because ${\sum }_{i,j}{s}_i{s}_j=({\sum }_i{s}_i{)}^2=n^2{m}^2$. The probability mass function of this spin magnetization is consequentlyand zero elsewhere. Grouping the original partition function by the same count also giveswhich verifies normalization without counting any configuration twice.

The binomial theorem makes the sum of the proposed probability mass function equal to $(p+1-p{)}^N=1$; every term is nonnegative. Realize the associated binomial distribution as $K=B_1+\cdots +B_N$, where the $B_j$ are independent variables with the Bernoulli distribution of parameter $p$. Each has expected value $p$ and variance $p(1-p)$. Linearity of expectation and variance additivity for independent random variables yieldAt $p=0$ or $p=1$ this is a constant random variable; the same formulas apply, with the endpoint mass understood by continuity.

For fixed $k$ and sufficiently large $N$,The finite product and last factor tend to one, while $(1-\lambda /N{)}^N\to e^{-\lambda }$. Thus the Poisson limit theorem givesThese numbers form a probability mass function because they are nonnegative and their sum is $e^{-\lambda }{\sum }_{k\ge 0}{\lambda }^k/k!=1$, by the exponential series. They define the Poisson distribution of parameter $\lambda$. Only $N\ge \lambda$ is needed to make the binomial parameter valid, which is sufficient for the limit.

A sufficient statistic has a conditional distribution of the full data given its value that is independent of the unknown parameter. A minimal sufficient statistic is sufficient and is a function of every other sufficient statistic, up to null sets. For a dominated family with common positive support, the likelihood-ratio criterion for minimal sufficiency states that $T$ is minimal sufficient if $T(x)=T(y)$ holds exactly when $p_{\theta }(x)/p_{\theta }(y)$ is independent of $\theta$.

For $S={\sum }_i{X}_i$, the joint probability mass function isThe Fisher-Neyman factorization theorem proves sufficiency of $S$. For two samples, the likelihood ratio is $(1-1/\theta {)}^{S(x)-S(y)}$, independent of $\theta >1$ exactly when their sums agree. Thus $S$ is minimal sufficient. Any one-to-one transformation of it is also minimal sufficient, so chooseThis chosen estimator is an unbiased estimator, since each observation has mean $\theta$. If instead the statistic is reported as $S$, its expectation is $n\theta$, and as an estimator of $\theta$ it has bias $(n-1)\theta$ (zero only when $n=1$). Being a minimal sufficient statistic is unchanged by this rescaling; being an unbiased estimator is not.

Since $\mathbb{P}(X_1=0)=e^{-\lambda }$, the indicator $Y$ is an unbiased estimator of $\theta$. The joint probability mass function isThe Fisher-Neyman factorization theorem therefore makes $T={\sum }_i{X}_i$ a sufficient statistic. More explicitly, conditioning on $T=t$ gives a multinomial distribution with $t$ trials and cell probabilities $1/n$, so $X_1\mid T=t$ has the binomial distribution $\mathrm{Bin}(t,1/n)$. Consequently conditioning by the Rao-Blackwell theorem givesFor $n=1$, interpret the value at $T=0$ as $1$ and at $T>0$ as $0$; thus the same expression uses $0^0=1$ in this probability convention. For $\lambda =0$ all observations and $T$ are zero almost surely. For $n>1$ and $\lambda >0$, the reduction in variance is strict: given $T=1$, the event $X_1=0$ still has a nondegenerate conditional probability. Directly, using the probability generating function of $T\sim \mathrm{Poisson}(n\lambda )$, the new variance is $e^{-2\lambda }(e^{\lambda /n}-1)$.

Put $q=e^{-\lambda t}$ and propose $P_n=q(1-q{)}^{n-1}$. The initial condition follows from $q(0)=1$. Since $q^{\prime }=-\lambda q$,On the other hand,and expanding the bracket gives the same expression. The formula also sums to one by the geometric series, so it is the required probability mass function.

Put $p_k=P_X(k)$. The assumption $p_0=0$ ensures that $X-1$ still takes values in $\{0,1,\dots \}$, and its probability mass function at $k$ is $p_{k+1}$. Using the paper's unhalved ${\ell }^1$ convention for the total variation distance,

The conditional probability mass function is

For the direct half of the code-distribution correspondence, let $L$ be the length function of a binary prefix code and putThen $R$ is a probability mass function andThus every prefix code determines a distribution whose ideal description lengths do not exceed the codeword lengths.

Conversely, given a probability mass function $R$, setfor $R(x)>0$. Then $2^{-L(x)}\le R(x)$, so the Kraft inequality is satisfied. Its converse supplies a binary prefix code with these lengths, andIf real lengths are allowed, the ideal choice $L(x)=-{\log }_{2}R(x)$ satisfies Kraft with equality.

Write $p(x)=\mathbb{P}(X_1^n=x)$, $a(x)=p(x)W(x)$, andOn the support of $p$, define the probability mass functionFor any real length function satisfying the Kraft inequality, put $K={\sum }_x{2}^{-L(x)}\le 1$ and $R_L(x)=2^{-L(x)}/K$. The Gibbs inequality givesEquality holds for the ideal weighted lengthsThus the smallest average weighted description length isWhen $p$ has full support, the displayed $L_n^{\ast }$ attains this minimum. If some strings have zero probability, the same value is the infimum over finite lengths and is attained by the extended-real ideal assignment $L_n^{\ast }(x)=\infty$ there; finite codewords of arbitrarily large length approach it.

The mean extension in an ensemble of $A$ molecules iswith constraints ${\sum }_i{a}_i=A$ and ${\sum }_i{a}_i{l}_i=AL$. For occupancies $\{a_i\}$, the number of assignments of the $A$ distinguishable ensemble members to the states is the multinomial coefficientThus maximizing the probability at fixed $A$ and $L$ is equivalent to maximizing the stated Lagrange multiplier expression.

Because $A\gg 1$, the Stirling formula gives $\log (a_i!)\simeq a_i\log a_i-a_i$. Differentiating with respect to each $a_i$ givesNormalization by ${\sum }_i{a}_i=A$ therefore yields the probability mass functionHere $Z$ is the fixed-tension partition function that normalizes the probabilities.

A Bernoulli source is a sequence $(X_n{)}_{n\ge 1}$ of independent and identically distributed random variables with a common probability mass function on the finite alphabet $\mathcal{A}$.

The source is reliably encodable at rate $r$ if, for each block length $n$, there are an encoder with at most $2^{nr}$ outputs and a decoder such that the block error probabilitytends to zero as $n\to \infty$.

Its information rate is the limiting information entropy per source symbol,when this limit exists.

Let $P\ne Q$ be probability mass functions on a finite alphabet, with $P$ absolutely continuous with respect to $Q$. A decision region $B_n$ accepts the null law $P^{\otimes n}$. Writefor its type-I and type-II errors. Stein's lemma states that for every fixed $0<\epsilon <1$,where logarithms and relative entropy use base two.

For achievability, letUnder $P^{\otimes n}$, the normalized log-likelihood ratio converges in probability to $D(P\Vert Q)$ by the weak law of large numbers, so $alph{a}_n\to 0$. On $B_n$, $Q^{\otimes n}\le 2^{-n(D(P\Vert Q)-\eta )}{P}^{\otimes n}$, whence ${\beta }_n\le 2^{-n(D(P\Vert Q)-\eta )}$.

For the converse, letAgain $P^{\otimes n}(A_n)\to 1$. If $alph{a}_n\le \epsilon$, thenThe final factor has positive lower limit at least $1-\epsilon$. Taking exponential rates and then $eta\downarrow 0$ proves the converse and the lemma.

Let $Z={\mathbb{E}}_Q{e}^g$ and define the exponentially tilted probability mass functionFor this choice every ratio $e^{g(x)}Q(x)/P^{\ast }(x)$ equals $Z$, so equality holds in the Jensen inequality used in part (d). Henceand $P^{\ast }$ is the maximizer. This is the finite-alphabet Gibbs variational principle for relative entropy.

Put $q=1-p$. Since $X$ takes positive integer values, $\mathbb{P}(X>0)=1=q^0$. Taking differences of the given tail probabilities therefore yields, also for $k=1$,This is the probability mass function of the geometric distribution on $\{1,2,\dots \}$.

For the geometric distribution, $\mu =1/p$, so the probability generating function of the size-biased distribution isIf $X_1,X_2$ are independent and identically distributed with geometric distribution of parameter $p$, the probability generating function of an independent sum givesAt zero, interpret the expression by its continuous extension. The probability generating functions agree, henceOne can also check the probability mass functions directly: for $k\ge 1$, there are $k$ positive integer pairs with sum $k+1$, each having probability $p^2{q}^{k-1}$. Thus both sides assign probability $k{p}^2{q}^{k-1}$ to $k$.