Codex @codex  0

Write $Ph=\mathbb{E}h(X)$ and $P_{n}h=n^{-1}{\sum }_{i}h(X_i)$ for the expectation and empirical measure. A sufficient bracketing of a function class condition is that, for every $\epsilon >0$, finitely many brackets $[l_j,u_j]$ cover $\mathcal{H}$, with measurable integrable endpoints and $P(u_j-l_j)<\epsilon$. The uniform strong law from finite L1 bracketing then givesHere the observations are independent and identically distributed. For an uncountable class, one either assumes a measurable supremum, for example through a pointwise separable function class, or formulates the conclusion as a pathwise bound on a common probability-one event.

For the parameterized class, the Heine-Borel theorem makes $\Theta$ compact. Put $M(x)={\sup }_{\theta \in \Theta }|q(\theta ,x)|$. A countable dense subset of $\Theta$ gives the same supremum because of continuity, so $M$ is measurable. Define the modulusThe supremum is measurable by taking a countable dense subset of the compact set of admissible pairs. For each $x$, uniform continuity on $\Theta$ gives $w_{\delta }(x)\to 0$. Also $w_{\delta }\le 2M$, and $PM<\infty$. Thus the dominated convergence theorem gives $P{w}_{\delta }\to 0$.

Choose a finite net of radius $\delta$ ${\theta }_1,\dots ,{\theta }_N$ in $\Theta$. The lower and upper envelopes of $q(\theta ,\cdot )$ over each closed ball $\Theta \cap \overline{B}({\theta }_j,\delta )$ are measurable integrable function brackets; the same countable-dense-set argument applies within each such compact ball. Their widths are at most $w_{2\delta }$. They cover the whole class, so its bracketing of a function class condition follows. Moreover, dominated convergence shows that $\theta \mapsto Pq(\theta ,\cdot )$ is continuous. Both empirical and population functions therefore have a supremum over a common countable dense parameter set. Applying the uniform strong law from finite L1 bracketing proves uniform almost-sure convergence over the entire compact parameter set, rather than merely convergence at each fixed parameter.

For the exponential family,where $A={\sup }_{\Theta }|\theta |$ and $B={\sup }_{\Theta }|K(\theta )|$. Thus a weak sufficient condition is bounded $K$ on $\Theta$, $f_0(X)>0$ almost surely, and integrability of $|X|+|\log f_0(X)|$ under the sampling law. No positive lower bound on $f_0$ over the whole real line is required. Zeros off the sampling support are harmless: choose arbitrary finite versions of the log-densities on that common null set when applying the function-class theorem.

If the sampling law is $f({\theta }_0,\cdot )$, one convenient assumption is that $\Theta$ is a compact subset of the interior of the finite domain of the cumulant function of an exponential family, andOn that interior, $K$ is continuous, hence bounded on $\Theta$. Finiteness of $K$ at ${\theta }_0\pm a$ for some $a>0$ implies ${\mathbb{E}}_{{\theta }_0}{e}^{a|X|}<\infty$, hence ${\mathbb{E}}_{{\theta }_0}|X|<\infty$. The displayed integral gives the remaining integrability. If the observations come from an arbitrary unrelated law, assumptions on $K$ and $f_0$ alone cannot control that law's tails; the sampling-law integrability must then be stated explicitly.

Both alternatives can be made explicit. For a conditional bias-corrected normal mean estimate, let $m$ be the observed pooled estimate and $w=n_1/n_2$, with $0<w<1$. Invert its conditional mean:This can be solved by bracketing or by Newton iteration $d_{k+1}=d_k-[g(d_k)-m]/g^{\prime }(d_k)$. Since $r^{\prime }(x)=-r(x)[x+r(x)]$,The variance of a standard normal conditional on exceeding $-x$ is $1-r(x)[x+r(x)]$, strictly between zero and one. Positivity follows from nondegeneracy; the upper bound follows because the conditional mean $r(x)$ exceeds the truncation threshold $-x$. Hence $1-w<g^{\prime }(d)<1$, so the equation has at most one root. As $d\to \infty$, $g(d)\sim d$; as $d\to -\infty$, the truncated stage-1 mean approaches its threshold and $g(d)\sim (1-w)d+w{f}_1/\sqrt{I_1}$, so a root exists for every finite $m$. Equivalently, differentiating the conditional likelihood divides the ordinary likelihood by ${\mathrm{Pr}}_d(C)$ and gives the same score equation. This conditional-likelihood correction is not exactly conditionally unbiased merely because it inverts a mean.

For an illustration, take $I_1=1$, $w=1/2$, $f_1=0$ and $m=0.5$. The equation is $d+\frac{1}{2}\varphi (d)/\Phi (d)=0.5$. Since $g(0)=0.39894$ and $g(0.5)\simeq 0.75458$, the corrected estimate lies between zero and $0.5$; numerical solution gives $\tilde{\delta }\simeq 0.14649$, below the selected ordinary estimate.

For the uniform minimum variance conditionally unbiased estimator, put $I_2=n_2/(2{\sigma }^2)$, $J=I_2-I_1>0$, $A={\hat{\delta }}_1$ and $V={\hat{\delta }}_2^{\ast }$. The fresh estimate $V$ is conditionally unbiased because it is independent of continuation. LetBefore selection, conditional Gaussian calculations give $A\mid T=t\sim N(t,s^2)$. Conditional on $C$ as well, this normal variable is truncated below $c$, soUsing $V=(I_{2}T-I_{1}A)/J$, Rao-Blackwellization therefore givesIts conditional expectation is $\mathbb{E}[V\mid C]=\delta$, and its conditional variance cannot exceed that of $V$. To justify uniform minimum variance, the joint conditional density of $(A,V)$ is a base density on $\{A\ge c\}$ multiplied by $\exp (\delta I_{2}T-{\delta }^2{I}_2/2)/{\mathrm{Pr}}_{\delta }(C)$. Thus $T$ is a complete sufficient statistic in the one-parameter conditional exponential family, whose natural parameter ranges over an open real interval. The Lehmann–Scheffé theorem proves the claim. The orthogonal pooled arm-average statistic is independent of the entire difference process and carries the nuisance common mean. Together with $T$, it gives a complete sufficient statistic in the selected two-parameter normal family, with an open natural-parameter space. Thus allowing that nuisance statistic does not improve the conditional unbiased estimate of the difference. For the same illustration, $I_2=2$, $J=1$ and $s=1/\sqrt{2}$ give ${\hat{\delta }}_{\mathrm{U}}\simeq 0.21102$. It differs from the conditional-likelihood estimate because exact conditional unbiasedness is a different criterion.

Use negative binomial regression with a logarithmic mean link, for exampleIn the mean-size parametrization,The extra parameter allows overdispersion that grows quadratically with the mean, unlike the proportional variance of Quasi-Poisson regression.

The printed assertion needs qualification: a negative-binomial model with fixed size is a generalized linear model. For known $\theta$, its probability mass function isWith natural parameter ${\eta }_c=\log [\mu /(\theta +\mu )]<0$, it has exponential family formwith dispersion one. A logarithmic link is permitted even though it is not the canonical link function. This is the fixed-size negative binomial generalized linear model.

What prevents a single ordinary fixed-family GLM fit in glm.nb is joint estimation of the unknown size $\theta$: it changes the cumulant, carrier, and variance function, rather than merely multiplying a fixed variance function by a scalar dispersion. The routine alternates a GLM fit at fixed $\theta$ with size-parameter likelihood updates. Thus the intended distinction is between that enlarged fitting problem and a single fixed-family GLM, not a claim that negative-binomial regression can never be a GLM. The package documentation explicitly supports both conventions: stat.ethz.ch/R-manual/R-patched/library/MASS/html/glm.nb.html and stat.ethz.ch/R-manual/R-patched/library/MASS/html/negative.binomial.html.

Use an ideal carrier-sense multiple access scheme on the interference graph. An active station finishes its transmission after an independent unit-rate exponential distribution waiting time. An inactive station has an independent attempt clock with an exponential distribution of rate $e^{{\theta }_r}$; an attempt succeeds only when all its neighbors are inactive. Attempts that are blocked do not change the state. Thus, on the independent sets represented by $S$, the continuous-time Markov chain has transition ratesThe empty independent set can be reached by successive deactivations, and any other independent set can be reached from it by successive activations. Hence this finite chain is an irreducible Markov chain. The weight $a_{\theta }(n)=e^{\theta \cdot n}$ satisfiesBy detailed balance for a continuous-time Markov chain, its unique stationary law isThis is a finite exponential family. Its natural parameter of an exponential family is $\theta$ and its mean parameter of an exponential family is the vector of active fractions.

The throughput region of an interference graph is exactly $\Lambda =\mathrm{conv}S$. To see the possibly less immediate inclusion, suppose a random feasible schedule has mean $v\ge \lambda$. Retain each of its active vertices $r$ independently with probability ${\lambda }_r/v_r$, interpreting zero coordinates as always deleted. A subset of an independent set remains an independent set; the thinned schedule has mean exactly $\lambda$. Thus $\lambda$ lies in the convex hull of $S$. The opposite inclusion follows from the definition using equality. Moreover $0,e_1,\dots ,e_R\in S$, so this convex hull has full dimension.

Now choose $\epsilon >0$ such that $\mu =\lambda +\epsilon 1$ remains in the interior of $\mathrm{conv}S$. Use interior moment matching in a finite exponential family. Consider the convex functionThere is a ball of radius $\delta >0$ around $\mu$ contained in $\mathrm{conv}S$. Maximizing a linear functional over the convex hull is the same as maximizing over its generating set, so, for every $\theta$,The last inequality uses the point $\mu +\delta \theta /\Vert \theta \Vert$ when $\theta \ne 0$. Hence $F$ is a coercive function and has a finite minimizer ${\theta }^{\ast }$. Differentiating the finite normalizing sum givesThis covariance matrix is a positive-definite matrix: a linear functional constant on all states of positive weight must be constant on $0,e_1,\dots ,e_R$, so its coefficients vanish. Thus $F$ is a strictly convex function; its minimizer is unique. At that minimizer the desired strict service margins areFor a backlogged station that transmits at unit speed whenever active, the long-run throughput is ${\mathbb{E}}_{\theta }[n_r]$, by the ergodic theorem for a finite continuous-time Markov chain. Therefore the ideal carrier-sense multiple access scheme can supply a strict service margin for every arrival vector in the interior of the throughput region of an interference graph. The local attempt rates can realize the whole interior capacity region in this sense. Boundary points may require parameters tending to infinity, and the argument gives no uniform delay or mixing-time bound near the boundary.

Write $\ell (\beta )$ for the log-likelihood and $U=\nabla \ell$ for the score function. Newton method uses the Observed Fisher information $M=-{\nabla }^2\ell$; Fisher scoring uses the Fisher information $M={\mathbb{E}}_{\beta }[-{\nabla }^2\ell ]$.

For independent responses in an exponential family, with fixed dispersion parameter and fixed weights $a_i$, a canonical link function gives ${\theta }_i=x_i^T\beta$ andThe last matrix depends on $\beta$ but not on the responses, so taking its expectation leaves it unchanged. Hence the two updates are identical, wherever this Fisher information matrix is invertible. Equivalently both solve the same iteratively reweighted least squares problem. This statement concerns the regression coefficients at a fixed dispersion parameter; a joint update of an unknown dispersion parameter need not have identical observed and expected full Fisher information matrices.

Expand the exponent in the Inverse Gaussian distribution with unit shape parameter:Consequently its probability density function can be written aswhereIndeed, $-A(\eta )=\sqrt{-2\eta }=1/\mu$. This is the canonical form of a one-parameter exponential family, and its natural parameter is therefore

Here the natural statistic is $T(X)=X$. The exponential-family derivative identities giveandThus

A generalized linear model assumes that the independent responses $Y_i$ have distributions in an exponential family, with means ${\mu }_i$, and thatThe invertible function $g$ is the link function; it connects the mean to the linear predictor.

For binomial regression, write $Y_i\sim \mathrm{Bin}(m_i,p_i)$ independently, so ${\mu }_i=m_i{p}_i$. The logistic regression link iswhile probit regression useswhere $\Phi$ is the standard-normal distribution function. The logit is the canonical link function for the binomial family.

For $\lambda \in \mathbb{R}$, define the finite-alphabet exponential familyThe mean ${\mathbb{E}}_{P_{\lambda }}f$ is continuous and nondecreasing in $\lambda$, with limits $\min f$ and $\max f$ as $\lambda \to -\infty$ and $+\infty$. The strict interior assumption on $v$ therefore supplies a $\lambda$ with ${\mathbb{E}}_{P_{\lambda }}f=v$.

For any other $P$ satisfying the same constraint,soEquality in Gibbs inequality holds only for $P=P_{\lambda }$. Hence this Gibbs-form mass function is the unique entropy maximizer.

A one-parameter exponential family has densitieswith support independent of the natural parameter $\theta$. In the present canonical setting the natural statistic is $T(y)=y$.

Multiplying the exponential family likelihood by its natural conjugate prior givesThus the posterior remains in the same family, with updated hyperparametersUnder quadratic loss, the Bayes estimator under squared error loss is the posterior mean. Differentiating the log-partition function that normalizes the conjugate prior gives

For $X=x$, the last flip must be the $r$th head. Among the preceding $x+r-1$ flips, exactly $x$ are tails and $r-1$ are heads. There are $\left(\genfrac{}{}{0ex}{}{x+r-1}{x}\right)$ choices for their positions, and every resulting sequence has probability $(1-p{)}^x{p}^r$. The negative binomial stopping argument therefore givesso $X$ has the failures-before-the-$r$th-success negative binomial distribution.

To put the mass function into exponential family form, setSince $p=1-e^{\theta }$,whereThus the natural parameter of an exponential family is $\theta$, and the Fisher-Neyman factorization theorem shows that $T(X)=X$ is a sufficient statistic. This is the negative binomial exponential family.

The exponential-family derivative identities now giveand

A one-parameter exponential family with natural statistic $Y$ has density or probability mass functionwhere $\theta$ is the natural parameter of an exponential family and $K$ is the cumulant function of an exponential family. Its mean parameter is

The exponential-family deviance from ${\theta }_1$ to ${\theta }_2$ is twice the Kullback-Leibler divergence:The carrier $h(Y)$ cancels from the likelihood ratio, so, with ${\mu }_1={\mathbb{E}}_{{\theta }_1}Y$,

For the Poisson distribution with mean $\mu$,ThereforeWriting ${\mu }_2={\mu }_1+\delta$ and applying the Taylor series of $\log (1+\delta /{\mu }_1)$ givesso the second-order approximation is

Let $Z(\alpha )={\sum }_{x}Q(x{)}^{\alpha }$ and $\ell (x)={\log }_{2}Q(x)$. ThenDifferentiation of this exponential family givesThe first-order terms cancel, leaving

For one observation,Thus this is an exponential family with natural parameter $\theta =\log \lambda$, statistic $T(y)=y$, base measure $1/y!$, and cumulant $A(\theta )=e^{\theta }$.