Past exam of the Probability, statistics and options II course of the University of Oxford / 2011 / M / M4

The moment-generating function is $M_X(t)=\mathbb{E}[e^{tX}]$ for real $t$ where this expectation is finite. Usually one requires finiteness on a neighbourhood of zero when using it to characterize a distribution. The characteristic function isIt always exists because the integrand has absolute value one. In both definitions the expectation is taken with respect to the distribution of $X$.

A characteristic function is useful for proving convergence in distribution, including for laws without a finite moment-generating function: the continuity theorem says that pointwise convergence of characteristic functions to a characteristic function continuous at zero implies convergence in distribution to its law. For example, the Cauchy distribution has a characteristic function although its moment-generating function is not finite on any neighbourhood of zero.

A moment-generating function is especially useful for exponential tail bounds: for $t>0$ where it is finite, Markov's inequality givesOptimizing over $t$ yields a Chernoff bound. The positive real exponential in this argument permits the inequality; a unit-modulus complex exponential does not give the corresponding tail bound.

Expand the analytic factor around zero by the binomial series:Every coefficient is nonnegative. Adding $2/3$ to the constant coefficient gives probabilities $p_j$ with sum $g(1)=2/3+1/3=1$. Define $\mathrm{Pr}(T=j)=p_j$ for $j\ge 0$. This is a nonnegative integer-valued random variable and its probability generating function is exactly $g$. The power series has radius $14/5>1$, so the evaluations and derivatives used below are finite.

For a probability generating function, $\mathbb{E}T=g^{\prime }(1)$ and $\mathbb{E}[T(T-1)]=g^{\prime \prime }(1)$. HereThus $\mathbb{E}T=5/54$ and $\mathbb{E}[T(T-1)]=25/324$. Adding the first moment gives $\mathbb{E}{T}^2=55/324$, hence

Markov's inequality states that for a nonnegative random variable $Z$ and $a>0$,It follows immediately from $Z\ge a{1}_{\{Z\ge a\}}$ and monotonicity of expectation. A finite expectation makes the bound informative.

By the definition of the probability generating function,The evaluation at two is valid because two is inside the power-series radius $14/5$.

The event $T\ge 5$ is the same as $2^T\ge 32$. Apply Markov's inequality to the nonnegative random variable $2^T$ and use part (iv):The strict numerical comparison is enough even though the probabilistic inequality itself is non-strict.

At each index the two independent uniform symbols match with probability $p=k/k^2=1/k$. The match indicators are independent across indices. Consequently the waiting times between successive matches are independent geometric random variables on $1,2,\dots$ with success probability $p$, and $S_m$ is their sum.

For one waiting time $G$, summing the geometric series gives $\mathbb{E}{e}^{tG}=p{e}^t/[1-(1-p)e^t]$. Independence yieldsFor $k>1$ the moment-generating function is finite for $t<\log (k/(k-1))$. When $k=1$, every index matches, $S_m=m$ deterministically and the formula reduces to $e^{mt}$ for every real $t$.

For $t\ge 0$, the geometric survival probability givesFor $t<0$, the distribution functions are zero, as is the limiting exponential distribution function. Since that limiting distribution function is continuous everywhere, these limits proveThe parameter is rate one, so the limiting mean is one, consistent with $\mathbb{E}(S_1/k)=1$.