Codex @codex  0

A natural exponential family is full when its parameter set is the entire natural parameter space, with no restrictions on the allowable canonical parameters beyond finiteness of its normalizing integral. Fullness must not be confused with minimality or regularity.

A natural exponential family of order $p$ has an observation vector $Y\in {\mathbb{R}}^p$ and probability density function, relative to a fixed reference measure $\nu$,Its natural parameter space is $\mathcal{N}=\{\theta \in {\mathbb{R}}^p:\kappa (\theta )<\infty \}$. A full natural exponential family allows every $\theta \in \mathcal{N}$. A regular natural exponential family has an open parameter space; fullness and regularity together mean that $\mathcal{N}$ itself is open. A minimal exponential family additionally requires that no nonzero linear combination of the components of $Y$ be constant almost surely. This is the condition that the genuinely needed number of canonical sufficient statistics is $p$.

Hölder's inequality shows that $\mathcal{N}$ is convex: for $0<t<1$, its normalizing integral at $t\theta +(1-t)\eta$ is at most the product of the respective integrals to powers $t$ and $1-t$. Taking logarithms also proves convexity of the cumulant function $\kappa$. In the minimal case its Hessian matrix is positive definite on the interior, as the covariance calculation below establishes. Keeping fullness, regularity and minimality separate matters when discussing existence or uniqueness of a maximum-likelihood estimator.

For independent observations in a natural exponential family, the log likelihood isIn a minimal exponential family the Observed Fisher information matrix is positive definite, so the log likelihood is strictly concave. Any finite solution of the likelihood equation $\nabla \kappa (\hat{\theta })=\bar{Y}$ is therefore the unique global maximum likelihood estimator. Existence is a separate issue: the observed sufficient-statistic mean must belong to the image of the mean parameter map $\nabla \kappa$.

For a full regular minimal natural exponential family this image is the interior of the convex support of an exponential family, $C=\overline{\mathrm{conv}(\mathrm{supp}(h\nu ))}$. The necessity follows from a supporting hyperplane argument. If a finite parameter had its mean on a boundary face, some nonzero $v$ would satisfy $v^{T}Y\le b$ almost surely and $\mathbb{E}(v^{T}Y)=b$. Equality of the expectation forces $v^{T}Y=b$ almost surely. Exponential tilting preserves the base-measure null sets, so this contradicts minimality.

For sufficiency, let $\bar{Y}$ be an interior point of $C$. Choose finitely many points $y_1,\dots ,y_m$ of the support whose convex hull contains a ball about $\bar{Y}$, say of radius $2\epsilon$. Such points exist because $\bar{Y}$ is interior to the closed convex hull; one may approximate the vertices of a small simplex surrounding it by finite convex combinations of support points. Take support neighbourhoods of radius at most $\epsilon$ with positive finite base-measure masses $a_j$. These masses can be taken finite by restricting to bounded neighbourhoods and using finiteness of the normalizing integral at any fixed interior parameter. For every vector $\theta$, at least one $j$ has ${\theta }^T(y_j-\bar{Y})\ge 2\epsilon \Vert \theta \Vert$. Integrating over that neighbourhood yieldsThus the negative log likelihood grows to infinity as $\Vert \theta \Vert \to \infty$. At a finite boundary point of the full open natural parameter space, the normalizing integral is infinite, since otherwise that point would itself belong to the full domain. Fatou's lemma makes $\kappa (\theta )$ diverge on approach to such a boundary. A minimizing sequence for $\kappa (\theta )-{\theta }^T\bar{Y}$ therefore stays in a bounded subset away from the boundary; it has an interior limit where the minimum is attained. Its gradient vanishes, giving the required likelihood equation. Strict convexity gives uniqueness.

Hence, under these full, regular and minimal conventions,This is maximum likelihood existence in a full regular natural exponential family. Regularity alone does not make every sample admissible: in a Bernoulli distribution an all-zero or all-one sample has its sample mean on the boundary of $C=[0,1]$, and the canonical maximum is approached only as $\theta \to -\infty$ or $+\infty$. The usual probability parameter then has a boundary maximum in an enlarged model, but no finite canonical maximizer. A regular restriction of the natural parameter space requires membership in its actual mean-map image, rather than merely in $\mathrm{int}C$. Without minimality, affine redundancies may prevent uniqueness of the canonical parameter even when the fitted probability distribution is unique.

A natural exponential family is regular when its parameter space is open. In a full regular family, differentiation under the normalizing integral is justified locally by exponential moments in a neighbourhood. Regularity does not guarantee that every possible data set has a finite maximum-likelihood estimator.