Codex @codex  0

For independent geometric random variables supported on $1,2,\dots$, put $S={\sum }_i{x}_i$. Multiplication of their probability mass functions gives the likelihood functionThere are $n$ terminal successes and $S-n$ preceding failures.

With no deletion states allowed, each match state must emit once in every aligned sequence. Thus the four gap-free alignment columns, $1,2,5,7$, become $M_1,M_2,M_3,M_4$. Columns $3$ and $4$ form one optional run, represented by a single looping insertion state $I_2$; column $6$ requires a separate insertion state $I_3$ after $M_3$. Two insertion states are sufficient and minimal in an ordered profile hidden Markov model: the two optional regions are separated by a compulsory emission and occupy different positions in the profile.

Estimate each transition probability by the number of observed uses of that transition divided by all transitions leaving its source state. The complete arrow diagram is given below; repeated labels denote the same state, so the second and third lines supply the branches of the first line. Start and End are silent:Three aligned sequences visit $I_2$, giving three exits to $M_3$, while one sequence emits twice there, giving one self-transition. Thus its exit and loop probabilities are $3/4$ and $1/4$, respectively. Two sequences visit $I_3$ and two bypass it. All absent transitions have probability zero.

The probability mass functions of the emitting states, estimated from all characters assigned to each state with no added counts, areIn particular, pooling the two optional columns into $I_2$ pools four emissions: two T, one A and one G. This is a generative Hidden Markov model, so it can generate other run lengths and combinations as well as the training sequences; reproducing each training column deterministically is not its purpose.

Fix any starting village and let $A$ be the pickup distance, $B$ the subsequent passenger distance. From each vertex, the four possible pickup distances are $0,5,10,15$, each with probability $1/4$. From each pickup village, the three other possible destinations have distances $5,10,15$, each with probability $1/3$.

Sharing the pickup village does not by itself give independence. Here it does hold because the conditional distribution of $B$ is the same for every pickup village: for all allowed $a,b$,Thus $D=A+B$ has the taxi distance convolution on a rectangular road cycle. Counting the twelve equally likely pairs givesNo other distances are possible. The probability distribution is the same for every starting village, so no assumption about the previous customer's destination is needed.

The separate moments are $\mathbb{E}A=15/2$, $\mathbb{E}B=10$, $\mathrm{Var}A=125/4$ and $\mathrm{Var}B=50/3$. Independence therefore givesAs a check from the displayed probability mass function, $\mathbb{E}{D}^2=2125/6$ and subtracting $(35/2{)}^2$ gives the same variance.

For a dominated family of sample distributions, a statistic $T$ is a sufficient statistic precisely when the probability mass function or density factors aswhere $h$ does not depend on the parameter. This is the Fisher-Neyman factorization theorem.

Here is the discrete proof in both directions. Suppose the factorization holds, and write $H(t)={\sum }_{x:T(x)=t}h(x)$. Then ${\mathbb{P}}_{\theta }(T=t)=g_{\theta }(t)H(t)$. Whenever this probability is positive, $H(t)$ is finite and positive andis independent of $\theta$. Thus the conditional distribution of the whole sample given $T$ is parameter-free, which is the definition of a sufficient statistic.

Conversely, if $T$ is sufficient, let $q_t(x)$ denote this common conditional mass function on each possible fiber. ThenTake $g_{\theta }(t)={\mathbb{P}}_{\theta }(T=t)$ and $h(x)=q_{T(x)}(x)$ to obtain the factorization. Fibers impossible for every parameter can be assigned $h=0$; conditional probabilities are required only for fibers with positive probability.

For the independent Poisson sample the joint mass function isAll parameter dependence is throughso this is a one-dimensional sufficient statistic. The sample mean contains the same information, since $n$ is known.

Let the annual count be $Y_t=y_t$, with null mean ${\lambda }_{0t}$ and alternative mean $\theta {\lambda }_{0t}$. Dividing the two probability mass functions for the Poisson distributions cancels $y_t!$ and the common power of ${\lambda }_{0t}$:Consequently the score for the Poisson likelihood-ratio CUSUM isAt $\theta =2$, this becomesUnder the stated Poisson model this identity is exact, rather than an additional approximation. The approximation in a practical application concerns the mortality model and the use of an estimated expected count.

For $\theta >1$, the expected scores verify the direction of evidence:The first inequality follows from $\log \theta <\theta -1$; the second bracket is zero at one and has positive derivative $\log \theta$ above one.

Within the approximation that a factory has either zero or one accident, let $N$ be the accident count. The factories' independence gives $N\sim \mathrm{Binomial}(m,p)$. Given an accident, its cluster of claims has a Poisson distribution of mean $\lambda$, independently across factories. Its probability generating function is $G_X(z)=e^{\lambda (z-1)}$. Thus this is a binomial accident portfolio with Poisson clusters, withA factory can contribute no claims either because it has no accident or because its accident's Poisson cluster is empty. Hence the no-claim probability differs from the no-accident probability $q^m$.

For $0\le n\le m$, the accident probability mass function is $p_n=\left(\genfrac{}{}{0ex}{}{m}{n}\right)p^n{q}^{m-n}$. For $1\le n\le m$,Therefore its Panjer claim-count class parameters areAt $n=m+1$ the multiplier is zero, and afterwards both successive masses are zero, so the recurrence holds for all $n\ge 1$. Finally, $f_0=e^{-\lambda }$ and $G_N(z)=(q+pz{)}^m$ reproduce the same initialization $g_0=G_N(f_0)$.

A Poisson distribution with parameter $\lambda >0$ assigns probability mass functionThe exponential series makes these probabilities sum to one. Shifting the indices in the absolutely convergent sums givesHence $E{Z}^2={\lambda }^2+\lambda$ andFor independent random variables $Z_1,\dots ,Z_n$ of parameter one, let $S_n={\sum }_j{Z}_j$. Summing their product probability mass function over all nonnegative $k_1+\cdots +k_n=k$ givesThe final equality follows from the multinomial theorem applied to $(1+\cdots +1{)}^k$. Thus $S_n$ has the Poisson distribution of parameter $n$.

Each $Z_j$ has expectation and variance one, so the central limit theorem gives convergence in distributionZero is a continuity point of the standard normal cumulative distribution function. ThereforeThe discrete atom at $S_n=n$ causes no difficulty: the central limit theorem applies to the entire event with threshold zero.

For discrete random variables, write $p(u,v)$ for the joint probability mass function and $p_U(u)$ for its marginal. The conditional entropy is the average of the entropies of the conditional distributions:Conditional distributions at zero-probability values of $u$ can be chosen arbitrarily; they contribute nothing. Here $h$ denotes Shannon entropy, with one common logarithm base and $0\log 0=0$.

For $p(u,v)>0$, use $\log p(u,v)=\log p_U(u)+\log p(v\mid u)$ and sum with weight $-p(u,v)$. Summing out $v$ in the first term givesFor countable alphabets, the entropy contributions are nonnegative, so the equality remains valid with extended values by summing nonnegative terms. Applying this identity successively to the prefix $(X_1,\dots ,X_{i-1})$ and $X_i$ proves the chain rule for information entropy:For $i=1$, the conditioning tuple is empty and the term is $h(X_1)$.

For the subset inequalities, first suppose the marginal entropies are finite, so all joint entropies below are finite by the chain rule and the assumed conditioning reduces entropy property. Put $H=h(X_1,\dots ,X_n)$, and let $H_{-i}$ be the joint entropy with coordinate $i$ omitted. Apply the two-variable chain rule in the order $(X_{[n]\setminus \{i\}},X_i)$ to getThe inequality holds because the prefix conditioning set is contained in the set of all other coordinates. Summing over $i$ and using the chain rule givesThis proves Han's entropy inequality. In terms of the normalized subset entropy, $h_n^{(n)}=H/n$ and $h_{n-1}^{(n)}={\sum }_i{H}_{-i}/[n(n-1)]$, hence $\boxed{h_n^{(n)}\le h_{n-1}^{(n)}}$ for $n\ge 2$.

Apply the same result to each $k$-element coordinate set $S$. It givesWhen this is summed over all $k$-subsets, each fixed $(k-1)$-subset $T$ occurs once for each possible added coordinate, namely $n-k+1$ times. ThereforeSubstitute ${\sum }_{|S|=k}h(X_S)=k\left(\genfrac{}{}{0ex}{}{n}{k}\right)h_k^{(n)}$ and the analogous identity at $k-1$. The binomial coefficient identity $k\left(\genfrac{}{}{0ex}{}{n}{k}\right)=(n-k+1)\left(\genfrac{}{}{0ex}{}{n}{k-1}\right)$ cancels the common positive factor and givesThis is the monotonicity of normalized subset entropies, obtained from the local $k$-coordinate inequality by explicit counting.

If some marginal entropy is infinite, each subset average contains at least one subset including that coordinate. The entropy of such a subset is at least that marginal entropy, by the nonnegative conditional-entropy chain rule, so every $h_k^{(n)}$ is infinite. The inequalities then hold in the extended sense. Otherwise the finite-entropy proof applies. For independent random variables the averages are all $n^{-1}{\sum }_{i}h(X_i)$; repeated copies of one variable instead give $h_k^{(n)}=h(X_1)/k$, illustrating the entropy reduction caused by redundancy.

Let $T_i=X_i+Y_i$. Since the two counts are independent Poisson distributions with means ${\lambda }_i$ and $\beta {\lambda }_i$, their total is a Poisson distribution with mean $(1+\beta ){\lambda }_i$. For a fixed observed total $t_i=x_i+y_i$, divide the joint probability mass function by that of the total:Define $p=\beta /(1+\beta )$, so $1-p=1/(1+\beta )$. ThenFor independent patients, the paired Poisson conditional likelihood isEvery nuisance rate ${\lambda }_i$ cancels, while each patient's own baseline count level is retained through the conditioning total. A patient with total zero would contribute the constant one.

For these data, ${\sum }_i{x}_i=161$, ${\sum }_i{y}_i=23$, and ${\sum }_i{t}_i=184$. Maximizing this conditional likelihood givesThese are the fit under the model without a cure component, not the estimates of the mixture model in the later parts.

For minimizing an energy $F(\theta )$, simulated annealing at temperature $T>0$ uses the Boltzmann distribution ${\pi }_T(\theta )\propto e^{-F(\theta )/T}$. At each temperature run a Metropolis–Hastings algorithm or Gibbs transition preserving that law, then reduce $T$. With a symmetric proposal, accept ${\theta }^{\prime }$ with probability $\min \{1,e^{-[F({\theta }^{\prime })-F(\theta )]/T}\}$: downhill moves are accepted and uphill moves remain possible. Keep the best state encountered. As temperature decreases, equilibrium mass favors global minima, provided the target is proper and the relevant concentration hypotheses hold. An arbitrary finite cooling run does not by itself prove global convergence.

Let $s={\sum }_{i=1}^m{x}_i$, $\bar{x}=s/m$ and $S={\sum }_i(x_i-\bar{x}{)}^2$. For the Poisson distribution model the observations must be nonnegative integers. Its log-likelihood isFor $s>0$, differentiation gives ${\ell }_P^{\prime }=-m+s/\lambda$ and ${\ell }_P^{\prime \prime }=-s/{\lambda }^2<0$. If $s=0$, the likelihood is decreasing and the maximizer is the boundary value zero. For the normal distribution model write $v={\sigma }^2>0$, givingFor each $v$, this is maximized at $\mu =\bar{x}$, and differentiating there gives $v=S/m$. HenceThe normal variance estimate has denominator $m$, not $m-1$. A finite positive normal maximum requires $S>0$; if all observations coincide, the likelihood is unbounded as $\mu =\bar{x},v\downarrow 0$. The normal annealing calculation below assumes $S>0$.

To maximize likelihood, choose the energy $-\ell$, so the annealing law is $L^{1/T}$, not $L^{-1/T}$. This sign is important in the Poisson request: literally inserting a positive log-likelihood into a minimizing Boltzmann law would give a nonnormalizable probability density function growing like $e^{m\lambda /T}$ at infinity. We use likelihood-power annealing with the maximizing sign.

For the normal distribution model, take base measure $d\mu dv$. The joint annealing probability density function isCompleting the square gives the Gibbs sampling conditional for $\mu$. Matching the conditional $v$ probability density function to the supplied inverse-gamma distribution convention givesIntegrating out $\mu$ contributes a factor $\sqrt{v}$, so the marginal law isThus this joint Boltzmann law is proper for $T<m/3$. Its variance marginal and the normal conditional both concentrate at the stated estimates as $T\downarrow 0$. For example, for $T<m/7$ its marginal mean and variance areAlso ${\mathbb{E}}_T(\mu -\bar{x}{)}^2=(T/m){\mathbb{E}}_{T}v\to 0$.

There is a direct convergence proof for the actual changing-temperature Gibbs algorithm, beyond convergence of its equilibrium probability densities. Start at a finite deterministic $v_0>0$, choose deterministic $0<T_n\le m/10$ with $T_n\to 0$, draw ${\mu }_n$ conditional on $v_{n-1}$, then draw $v_n$ conditional on ${\mu }_n$ using the two boxed distributions. Conditional normal moments giveThe inverse-gamma distribution first and second moments consequently yieldandIn the first recursion the coefficient of the preceding mean is at most $1/6$; in the second the coefficient of the preceding second moment is at most $1/8$. The means therefore stay bounded, and then so do the second moments. Letting $T_n\to 0$ in these formulas gives $\mathbb{E}{v}_n\to S/m$ and $\mathbb{E}{v}_n^2\to (S/m{)}^2$. Together with $\mathbb{E}({\mu }_n-\bar{x}{)}^2=T_n\mathbb{E}{v}_{n-1}/m\to 0$, this provesThis Gaussian likelihood Gibbs annealing convergence uses the explicit conditional structure and does not assume that one Gibbs sampling sweep is an exact equilibrium draw.

For the Poisson distribution model, the likelihood-power law with respect to $d\lambda$ has the normalized probability density functionsoThe mean tends to $\bar{x}$ and the variance to zero, proving convergence to a point mass at the Poisson maximum-likelihood estimate, including the zero-boundary case.

The Akaike information criterion is $2k-2\ell (\hat{\theta })$, counting one parameter for Poisson and two for normal. With $0\log 0=0$ and $S>0$, the two forms areandChoose the smaller criterion. Likelihood constants that differ between models, including the factorial and $2\pi$ terms, cannot be discarded.

For a fully specified trans-dimensional annealing for penalized likelihood, let $M=P$ or $N$ and defineUse the combined target, relative to counting measure on models and Lebesgue measure on their parameters,where $w_P,w_N>0$ are fixed reference weights. For $S>0$ and $0<T<m/3$ both model components are integrable. The factor $2T$ is a convenient temperature convention keeping the within-model laws exactly those already derived. The extra factor $e^{-k_M/T}$ makes the zero-temperature objective AIC rather than unpenalized likelihood.

Use, for example, probability $1/2$ for a model-change attempt in either model and probability $1/2$ for a within-model update. Within the Poisson distribution model draw a fresh gamma value from its boxed law; within the normal distribution model perform the boxed Gibbs sampling sweep. For a Poisson-to-normal proposal, draw $u\sim q(u)=N(0,{\tau }^2)$ with a fixed ${\tau }^2>0$, and setThe inverse normal-to-Poisson proposal is $\lambda =v$ with reverse auxiliary $u=\mu -v$. This dimension matching uses one auxiliary variable to augment the one-dimensional Poisson parameter space. The absolute Jacobian determinant isIf the forward model-jump selection probability is $b$ and the reverse probability is $d$, the reversible-jump Markov chain Monte Carlo acceptance ratio isAccept with probability $\min (1,R_{P\to N})$. For the reverse move compute $\lambda =v,u=\mu -v$ and useFor the suggested equal selection probabilities, $b=d=1/2$ and those factors cancel. The probability density function $q$ must still be included, despite symmetry of its normal distribution: it is an auxiliary draw in only the forward dimension-increasing proposal. Pairing a move with its inverse makes the accepted forward and reverse probability flows equal after the change of variables, proving detailed balance for each fixed-temperature model-switch kernel.

Cool through proper temperatures, allow enough model and parameter exploration at each stage, and retain the smallest observed $E_M$. The minima of the two energies are exactly the two AIC values above. Low-temperature equilibrium mass concentrates on the better criterion and its fitted parameters; if they tie, the minimizing set contains both models. Unlike the direct normal Gibbs argument, this multimodel exploration claim needs an appropriate cooling/mixing justification and is not guaranteed by an arbitrary short run.

Finally, an exact statistical comparison must put both likelihoods on the same observable and reference-measure convention. A Poisson distribution model is a probability mass function on counts, whereas the displayed normal distribution model is a continuous probability density function. The formulas above are the question's formal Gaussian working-likelihood comparison. A coherent exact normal approximation for count observations can instead use probabilities of rounding intervals; that changes its likelihood and generally loses the simple analytic normal estimates. This distinction matters when interpreting the numerical AIC difference, while leaving the dimension-matching and annealing construction applicable to properly specified likelihoods.

Use base-two logarithms and the convention $0{\log }_{2}0=0$. Write $p_y=\mathbb{P}(Y=y)$ and $q_y(x)=\mathbb{P}(X=x\mid Y=y)$ for $p_y>0$. The conditional entropy is the average of the information entropies of these conditional laws:The careful form of Gibbs inequality is that, for two probability mass functions $q,p$ on a common countable set,where a term with $q(x)>0=p(x)$ gives $+\infty$. Equality holds exactly when $q=p$. This is the nonnegativity of relative entropy.

First suppose $H(X)<\infty$. For every $y$ of positive probability, $q_y(x)>0$ implies $p_X(x)>0$. Averaging the cross-entropies givesAll the summands here are nonnegative, so interchanging the sums is valid, and each conditional cross-entropy is finite. Applying Gibbs inequality to each $q_y,p_X$ shows that $H(q_y)$ is at most its cross-entropy. Subtracting and averaging therefore givesEvery term on the right is nonnegative. The equality in the entropy conditioning inequality is thus equivalent to $q_y=p_X$ for all $p_y>0$, which is exactly factorization of the joint probability mass function. Consequently

For arbitrary countable discrete random variables, the inequality remains valid in $[0,\infty ]$: the case $H(X)=\infty$ is immediate. The finite-entropy condition is necessary for the usual equality characterization. For example, let $\mathbb{P}(A=n)=c/(n(\ln n{)}^2)$ for $n\ge 2$, with $c$ the normalizing constant, and let $B$ be an independent fair bit. The series defining the probability mass function converges, whereas its information entropy diverges like ${\sum }_{n}1/(n\ln n)$. Then $X=(A,B)$ and $Y=B$ are dependent, but $H(X)=H(X\mid Y)=\infty$. In the infinite case, equality means precisely that the conditional entropy is also infinite; it does not imply independent random variables.

Introduce a selector $B$ with Bernoulli distribution of parameter $\alpha$, and construct $W$ by taking its conditional law given $B=1$ to be $p_U$, and its conditional law given $B=0$ to be $p_V$. No prescribed joint law of $U,V$ is needed. Summing over the selector gives the desired mixture probability mass function. The conditional entropy isSince conditioning cannot increase information entropy,This proves the concavity of information entropy. The argument also covers infinite information entropies with the convention that a zero mixing weight contributes zero. At $\alpha =0$ or $1$, the mixture is the corresponding original law and equality is immediate.

The claim-size exponential distribution has expected value $\mu >0$ and moment-generating functionConditional on $N_i=k$, independence makes the aggregate moment-generating function equal to $M_X(t{)}^k$. Averaging over the Poisson distribution of the count givesPut $\Lambda ={\sum }_{i=1}^n{\lambda }_i$. For fixed intensities, independence between risks givesThis is precisely the transform of a compound Poisson distribution. Indeed the addition of independent Poisson random variables makes the total count Poisson distributed with parameter $\Lambda$, and every claim in the pooled portfolio has the same exponential distribution. Thus the portfolio aggregate is compound Poisson with intensity ${\sum }_i{\lambda }_i$ and exponential claim sizes of mean $\mu$.

Now regard the individual intensities as independent gamma distributed quantities, with shape $m\ge 1$ and rate $\alpha$. Their moment-generating functions are $(\alpha /(\alpha -t){)}^m$ for $t<\alpha$. Multiplying these transforms proves additivity of independent gamma distributions with a common rate:Conditional on the individual intensities, the total count $N$ has Poisson distribution with parameter $\Lambda$. Because this conditional law depends only on their sum, it is also the law of $N$ conditional on $\Lambda$. Integrating its probability mass function against the gamma distribution density yieldsThus the total count has a negative binomial distribution with shape $nm$ and success probability $\alpha /(\alpha +1)$, using the convention that counts failures before the specified number of successes. Its probability generating function is $(\alpha /(\alpha +1-z){)}^{nm}$.

Conditional on $\Lambda$, the claim sizes retain their common exponential distribution and the amount $S$ has a compound Poisson distribution of intensity $\Lambda$. Therefore the unconditional amount has a compound mixed Poisson distribution with gamma mixing intensity of shape $nm$ and rate $\alpha$. To make the mixture explicit, average the conditional aggregate moment-generating function:The displayed domain ensures both the severity transform and the intensity transform are finite. The mixing distribution describes the Poisson intensity, not the claim sizes. In particular, the aggregate has an atom at zero of mass $\mathbb{P}(N=0)=(\alpha /(\alpha +1){)}^{nm}$.

Use Bayesian rejection sampling for a segregating-site count. Independently propose $\theta \sim \pi$ and $T_j\sim \mathrm{Exp}({\lambda }_j)$ for $j=2,\dots ,n$, and calculate $L={\sum }_{j}j{T}_j$. Draw an independent uniform $U$ on $(0,1)$ and accept the whole proposed vector exactly whenRepeat after a rejection. The right side is a probability mass function value of a Poisson distribution and is therefore between zero and one. On acceptance, the probability density function of a proposed vector is its prior density multiplied by this acceptance probability, divided by the probability of acceptance. Part (c) shows that this is precisely $f(\theta ,T\mid S=k)$. Independent proposals and uniforms produce independent posterior draws. Equivalently, one could simulate a count from a Poisson distribution with mean $\theta L/2$ and retain the proposal when that count equals $k$; the uniform version avoids simulating that extra count.

A nondegenerate distribution function $G$ is a max-stable distribution if, for every integer $r\ge 1$, there are $a_r>0$ and $b_r\in \mathbb{R}$ such thatEquivalently, the suitably centered and positively scaled sample maximum of $r$ independent and identically distributed random variables with law $G$ again has law $G$. The extremal types theorem states that every nondegenerate limiting law of normalized sample maxima is a max-stable distribution. Thus this property identifies all possible limiting types, although a particular starting distribution function need not admit any such limiting normalization.

The three standard types are the Gumbel distributionthe Fréchet distribution, with $\alpha >0$,and the negative Weibull distribution, with $\alpha >0$,They are max-stable distributions, as follows respectively by taking $(a_r,b_r)=(1,\log r)$, $(r^{1/\alpha },0)$, and $(r^{-1/\alpha },0)$. The necessary and sufficient classification is: $G$ is max-stable exactly when $G(x)=H((x-b)/a)$ for $a>0$, $b\in \mathbb{R}$, and one of these three types $H$. A positive affine map sends a max-stable distribution to another max-stable distribution.

For the requested threshold equivalence, set $p_n=1-F(u_n)$. By independence,If $n{p}_n\to \tau <\infty$, then $p_n\to 0$ and the expansion $\log (1-p)=-p+O(p^2)$ gives $n\log (1-p_n)\to -\tau$, since $n{p}_n^2=(n{p}_n)p_n\to 0$. Exponentiation proves the claimed limit. Conversely, if $(1-p_n{)}^n\to e^{-\tau }>0$, then $p_n\to 0$: any subsequence with $p_n\ge \delta >0$ would have its powers tending to zero. Taking logarithms now gives $-n\log (1-p_n)\to \tau$. Since $-\log (1-p)/p\to 1$, including the continuous extension at $p=0$, it follows that $n{p}_n\to \tau$. Hence

The exceedance count $S_n$ has the binomial distribution with parameters $n,p_n$. For $\tau >0$ and each fixed nonnegative integer $s$, its probability mass function satisfieswhere $(n{)}_s=n(n-1)\cdots (n-s+1)$ and $(n{)}_0=1$. Indeed $(n{)}_s/n^s\to 1$, $(n{p}_n{)}^s\to {\tau }^s$, and $(1-p_n{)}^{n-s}\to e^{-\tau }$. If $\tau =0$, the bound $\mathbb{P}(S_n\ge 1)\le \mathbb{E}{S}_n=n{p}_n\to 0$ gives the degenerate Poisson distribution at zero. Summing the finitely many terms proves, in both cases,This derives the needed instance of the Poisson limit theorem directly.