Past exam of the mathematics course of the University of Cambridge / 2016 / iii / Paper 201 / 3

Let $\mu =\mathbb{E}{X}_1$ and let the cumulant-generating function and its Legendre transform beUnder the permitted assumption that $\psi$ is finite everywhere, the right-tail form of the Cramér theorem is, for $a\ge \mu$,Use $\log 0=-\infty$. The restriction to nonnegative parameters is valid on this side of the mean: convexity gives $\psi (\theta )\ge \mu \theta$, so negative parameters cannot improve the value zero available at $\theta =0$. For $a<\mu$, the right-tail probability tends to one by the strong law of large numbers, so its logarithmic rate is zero; it is not generally $-I(a)$.

The full large deviation principle bounds probabilities of closed sets from above and open sets from below with rate $I$. For the displayed tail equality, a possible finite upper endpoint also needs attention. If $b=\mathrm{esssup}{X}_1<\infty$, then at $a=b$ the event requires every summand to equal $b$; its probability is $\mathbb{P}(X_1=b{)}^n$, and $I(b)=-\log \mathbb{P}(X_1=b)$. The identity follows by taking $\theta \to \infty$ in $\mathbb{E}{e}^{\theta (X_1-b)}$. Above $b$, both sides have infinite rate. The theorem describes exponential decay, including endpoints and impossible events.

For every $\theta \ge 0$, the Markov inequality and independence give the Chernoff boundOptimize over nonnegative $\theta$. For $a\ge \mu$, part (a) identifies this supremum with $I(a)$, yielding the finite-$n$ estimateIf $I(a)=\infty$, use parameters with arbitrarily large $\theta a-\psi (\theta )$; the probability must then be zero. Taking the logarithmic upper limit proves the upper bound in the Cramér theorem:For $a<\mu$, the nonnegative-parameter supremum is zero, giving the correct trivial tail-rate upper bound. Exponential Markov bounds and the product moment-generating function supply the entire upper-bound argument.

Write $b=\mathrm{esssup}{X}_1$. The hypothesis $\mathbb{P}(X_1>a)>0$ implies $a<b$ and rules out a constant random variable. In particular $I(a)<\infty$. The cumulant-generating function is strictly convex, since its second derivative is the positive variance under exponential tilting, and ${\psi }^{\prime }(0)=\mu$, ${\psi }^{\prime }(\theta )\to b$ as $\theta \to \infty$.

If $a>\mu$, there is a unique ${\theta }_a>0$ with ${\psi }^{\prime }({\theta }_a)=a$, and the Legendre transform gives $I(a)={\theta }_{a}a-\psi ({\theta }_a)$. ThusIf $a=\mu$, use differentiability at zero: $\psi (\theta )=\mu \theta +o(\theta )$, so for some sufficiently small $\theta >0$,In either case choose a parameter $\theta \ge 0$ with $J=\theta (a+\epsilon )-\psi (\theta )>I(a)$, and choose $0<\eta <J-I(a)$. The Cramér theorem gives, for all sufficiently large $n$,The Chernoff bound at the chosen parameter gives $\mathbb{P}(S_n/n\ge a+\epsilon )\le e^{-nJ}$. The conditioning event has positive probability for every $n$, since all summands can simultaneously exceed $a$. ThereforeTaking the complement within the conditioning event proves the conditional concentration at a large-deviation threshold:The farther tail has a strictly larger exponential cost. The argument also works when $a+\epsilon$ reaches or exceeds the upper support endpoint, without requiring an interior maximizer there.