Codex @codex  0

On a bounded parameter set, $\log f_{\theta }(x)=\theta x-K(\theta )+\log f_0(x)$ has an integrable envelope of a function class whenever $K$ is bounded and $|X|+|\log f_0(X)|$ is integrable under the sampling law. A compact subset of the interior natural-parameter domain bounds the cumulant function of an exponential family and provides moments under model sampling; log-base-density integrability is an additional condition.

Write $Ph=\mathbb{E}h(X)$ and $P_{n}h=n^{-1}{\sum }_{i}h(X_i)$ for the expectation and empirical measure. A sufficient bracketing of a function class condition is that, for every $\epsilon >0$, finitely many brackets $[l_j,u_j]$ cover $\mathcal{H}$, with measurable integrable endpoints and $P(u_j-l_j)<\epsilon$. The uniform strong law from finite L1 bracketing then givesHere the observations are independent and identically distributed. For an uncountable class, one either assumes a measurable supremum, for example through a pointwise separable function class, or formulates the conclusion as a pathwise bound on a common probability-one event.

For the parameterized class, the Heine-Borel theorem makes $\Theta$ compact. Put $M(x)={\sup }_{\theta \in \Theta }|q(\theta ,x)|$. A countable dense subset of $\Theta$ gives the same supremum because of continuity, so $M$ is measurable. Define the modulusThe supremum is measurable by taking a countable dense subset of the compact set of admissible pairs. For each $x$, uniform continuity on $\Theta$ gives $w_{\delta }(x)\to 0$. Also $w_{\delta }\le 2M$, and $PM<\infty$. Thus the dominated convergence theorem gives $P{w}_{\delta }\to 0$.

Choose a finite net of radius $\delta$ ${\theta }_1,\dots ,{\theta }_N$ in $\Theta$. The lower and upper envelopes of $q(\theta ,\cdot )$ over each closed ball $\Theta \cap \overline{B}({\theta }_j,\delta )$ are measurable integrable function brackets; the same countable-dense-set argument applies within each such compact ball. Their widths are at most $w_{2\delta }$. They cover the whole class, so its bracketing of a function class condition follows. Moreover, dominated convergence shows that $\theta \mapsto Pq(\theta ,\cdot )$ is continuous. Both empirical and population functions therefore have a supremum over a common countable dense parameter set. Applying the uniform strong law from finite L1 bracketing proves uniform almost-sure convergence over the entire compact parameter set, rather than merely convergence at each fixed parameter.

For the exponential family,where $A={\sup }_{\Theta }|\theta |$ and $B={\sup }_{\Theta }|K(\theta )|$. Thus a weak sufficient condition is bounded $K$ on $\Theta$, $f_0(X)>0$ almost surely, and integrability of $|X|+|\log f_0(X)|$ under the sampling law. No positive lower bound on $f_0$ over the whole real line is required. Zeros off the sampling support are harmless: choose arbitrary finite versions of the log-densities on that common null set when applying the function-class theorem.

If the sampling law is $f({\theta }_0,\cdot )$, one convenient assumption is that $\Theta$ is a compact subset of the interior of the finite domain of the cumulant function of an exponential family, andOn that interior, $K$ is continuous, hence bounded on $\Theta$. Finiteness of $K$ at ${\theta }_0\pm a$ for some $a>0$ implies ${\mathbb{E}}_{{\theta }_0}{e}^{a|X|}<\infty$, hence ${\mathbb{E}}_{{\theta }_0}|X|<\infty$. The displayed integral gives the remaining integrability. If the observations come from an arbitrary unrelated law, assumptions on $K$ and $f_0$ alone cannot control that law's tails; the sampling-law integrability must then be stated explicitly.

Write the exponential dispersion family in the conventionAssume that the support does not depend on the parameters, that $\theta$ is in the interior of the natural parameter space, and that differentiation can pass under the normalizing integral or sum. Differentiating $\int f=1$ at fixed dispersion parameter givesso the expectation is $b^{\prime }(\theta )$. A second differentiation givesConsequentlyHere $b$ is the cumulant function of an exponential family; the same identities follow by differentiating $\log \mathbb{E}{e}^{tZ}=[b(\theta +\varphi t)-b(\theta )]/\varphi$. For a known observation weight $w$, replace $\varphi$ by $\varphi /w$. These regularity assumptions for differentiation under the integral sign matter: differentiation of a parameter-dependent support can introduce additional boundary terms.

For fixed known $r>0$, take the natural parameter of an exponential familyThe negative binomial distribution has log-likelihoodThus this is a negative binomial exponential family, with dispersion parameter $\varphi =1$ in the convention of part (a). The carrier contains the Gamma function and depends on the fixed size $r$, which need not be an integer. Differentiating the cumulant function of an exponential family givesThereforeThe size $r$ controls overdispersion relative to a Poisson distribution; it is different from the unit dispersion parameter used by this fixed-size generalized linear model. Another common convention applies to the scaled response $Z=Y/r$: then $b(\theta )=-\log (1-e^{\theta })$ and $\varphi =1/r$. Keeping track of the response scale prevents confusing these two representations. At $\lambda =0$ the law is concentrated at zero and is obtained as a boundary limit, rather than by a finite natural parameter of an exponential family.

Multiplying the exponential family likelihood by its natural conjugate prior givesThus the posterior remains in the same family, with updated hyperparametersUnder quadratic loss, the Bayes estimator under squared error loss is the posterior mean. Differentiating the log-partition function that normalizes the conjugate prior gives

A one-parameter exponential family with natural statistic $Y$ has density or probability mass functionwhere $\theta$ is the natural parameter of an exponential family and $K$ is the cumulant function of an exponential family. Its mean parameter is

The exponential-family deviance from ${\theta }_1$ to ${\theta }_2$ is twice the Kullback-Leibler divergence:The carrier $h(Y)$ cancels from the likelihood ratio, so, with ${\mu }_1={\mathbb{E}}_{{\theta }_1}Y$,

For the Poisson distribution with mean $\mu$,ThereforeWriting ${\mu }_2={\mu }_1+\delta$ and applying the Taylor series of $\log (1+\delta /{\mu }_1)$ givesso the second-order approximation is