Probability course of the University of Oxford (A8)

Source: Oxford's released examination PDF, pages 172–175, dated 19 June 2023; questions 1, 2 and 3 occupy pages 173, 174 and 175 respectively. The solutions below follow the paper's numbering. In 2(b), both the original PDF and its transcription omit independence between $B$ and $U$; the intended independent-case answers and the consequences of this omission are distinguished below.

The strong law of large numbers states that, for independent and identically distributed random variables $X_1,X_2,\dots$ with $\mathbb{E}|X_1|<\infty$,Thus, outside an event of probability zero, the sample mean converges to the common expected value.

For a Borel set $A\subseteq \mathbb{R}$, the joint probability density givesFor each $x\ne 0$, substitute $z=xy$. The change of variables for probability densities contributes $dy=dz/|x|$, with the absolute value accounting for the reversed orientation when $x<0$. The line $x=0$ has two-dimensional Lebesgue measure zero, so omitting it changes no probability. By the Tonelli theorem,The bracket is nonnegative and its integral over $\mathbb{R}$ is one. This proves the probability density of a product of random variables formula:As usual, a probability density function is determined only almost everywhere; exceptional infinite values are harmless.

For the independent random variables with the specified uniform distribution, $f(x,y)=b^{-2}$ on $(0,b{)}^2$ and zero elsewhere. When $0<z<b^2$, the conditions $0<x<b$ and $0<z/x<b$ are equivalent to $z/b<x<b$. ThereforeThe value assigned at zero is arbitrary. Substitution $s=z/b^2$ checks normalization of a probability distribution: ${\int }_0^1-\log sds=1$. This is the product of independent uniform random variables law.

Direct integration of the uniform distribution yields $\mathbb{E}{U}_j=b/2$ and $\mathbb{E}{U}_j^2=b^2/3$. The expectation of a product of independent random variables givesSubtracting the square of the expected value gives the variance:Since $W_n\ge 0$, Markov's inequality implies, for every $\epsilon >0$,Hence $\boxed{W_n\to 0\text{ in probability for }0<b<2.}$ The first moment suffices: the variance need not tend to zero throughout this range.

If $0<b\le 1$, then $0<U_j\le 1$ almost surely. Intersecting these events over all $j$ still gives an event of probability one. On it, $0\le W_{n+1}\le W_n$, so the products have a finite nonnegative limit $W_{\infty }$.

For every $\epsilon >0$, the inclusion $\{W_{\infty }>\epsilon \}\subseteq \{W_n>\epsilon \}$ and part (c)(i) imply $\mathbb{P}(W_{\infty }>\epsilon )=0$. Taking $\epsilon =1/m$ for positive integers $m$ proves $W_{\infty }=0$ almost surely. ThusThe monotonicity is the extra ingredient that upgrades convergence in probability to almost sure convergence here.

For $\alpha >0$, integration and independence of random variables giveBy the fractional moment criterion for convergence in probability,The parenthesized factor is less than one precisely when $\alpha \log b-\log (1+\alpha )<0$. This expression vanishes at $\alpha =0$ and has derivative $\log b-1$ there. If $b<e$, the derivative is negative, so some sufficiently small $\alpha >0$ makes the factor strictly less than one. The bound then tends to zero geometrically, proving

The inverse transformation of a random variable is $u=e^{-v}$, with absolute derivative $e^{-v}$. The condition $0<u<b$ becomes $v>-\log b$. The change of variables for probability densities givesEquivalently, $V_n+\log b$ has the exponential distribution of rate one. Applying a separate measurable transformation of a random variable to each independent $U_n$ preserves independence of random variables, so the $V_n$ are independent and identically distributed.

Taking logarithms of the product givesBy part (d)(i), $V_j=E_j-\log b$ in probability distribution, where $E_j$ has the exponential distribution of rate one. Thus $\mathbb{E}|V_j|\le 1+|\log b|<\infty$ and $\mathbb{E}{V}_j=1-\log b$. The strong law of large numbers givesIf $b<e$, choose $\delta =(1-\log b)/2>0$. Eventually $\log W_n\le -\delta n$, so $W_n\le e^{-\delta n}\to 0$. If $b>e$, choose $\delta =(\log b-1)/2>0$; eventually $W_n\ge e^{\delta n}\to \infty$. ThereforeIn particular, the second range cannot have almost sure convergence to zero. This is the almost-sure growth rate of independent products. At $b=e$ the logarithmic limit is zero, which by itself decides neither behavior; that boundary case is not requested.

The uniqueness theorem for moment-generating functions states that if $M_X$ and $M_Y$ are finite and equal on an open real interval containing zero, thenFor the convergence theorem for moment-generating functions, suppose that the moment-generating functions $M_{X_n}$ are finite on a common open real interval $(-a,a)$, $a>0$, and $M_{X_n}(t)\to M_X(t)$ for every $|t|<a$, where $M_X$ is finite there. ThenOne may equivalently assume convergence to a finite function $M$ on that interval: the theorem then asserts existence of a unique limiting probability distribution with moment-generating function $M$ as well as convergence in distribution.

The independence of random variables $B$ and $U$ is needed for the intended mixture distribution in (ii) and (iii). It is absent from the printed assumptions. We use independence for the main formulas, and then give the answers under the literal assumptions.

The uniform distribution has probability density function $1/2$ on $(-1,1)$. Therefore, for $t\ne 0$,The value at zero follows either directly from the definition of the moment-generating function or by continuity. This answer needs no assumption about $B$.

If $B$ and $U$ are independent random variables, conditioning on the Bernoulli random variable $B$ giveswith the continuous value $M_Z(0)=1$. Indeed $Z=0$ when $B=0$, while the conditional probability distribution of $Z$ given $B=1$ is the original uniform distribution.

Under the literal assumptions alone, the law of total expectation gives onlyThis is not determined by $M_U$ and $p$. For an explicit counterexample, put $p=1/2$ and $B=1_{\{U>0\}}$. Then $B$ has the required Bernoulli distribution, butwhich differs from the independent-case formula.

Under independence of random variables, the law of a random variable $Z$ is the mixed probability distribution $(1-p){\delta }_0+p\mathrm{Unif}(-1,1)$. For negative $z$, only the uniform component can contribute to $\mathbb{P}(Z\le z)$; at zero the probability mass $1-p$ is included. Thus its cumulative distribution function isThe sketch consists of a horizontal segment at height zero before $-1$, a line of slope $p/2$ ending at the open point $(0,p/2)$, a jump of size $1-p$ to the closed point $(0,1-p/2)$, another line of slope $p/2$ ending at $(1,1)$, and a horizontal segment thereafter. The cumulative distribution function is right-continuous at the jump.

Under the literal assumptions, write $H(z)=\mathbb{P}(U\le z\mid B=1)$. The general answer is $F_Z(z)=(1-p)1_{\{z\ge 0\}}+pH(z)$, so the continuous part of the sketch is undetermined. The jump at zero is still exactly $1-p$, because $\mathbb{P}(U=0)=0$. In the dependent counterexample from (ii), $F_Z(z)=0$ for $z<0$, $F_Z(z)=(1+z)/2$ for $0\le z<1$, and $F_Z(z)=1$ for $z\ge 1$.

Fix $0<c<1$ and choose $R_c$, independently of $X$, with the normal distributionThe linear combination of independent normal random variables $cX+R_c$ is normal. Its expected value is $c\mu +(1-c)\mu =\mu$ and its variance is $c^2{\sigma }^2+(1-c^2){\sigma }^2={\sigma }^2$. Hence $cX+R_c\stackrel{d}{=}X$ for every $c$, proving that every specified normal distribution is a self-decomposable distribution.

Fix $|t|<t_0$. Since $|ct|<t_0$, $0<M_X(ct)<\infty$. Using independence of random variables and the Tonelli theorem for the positive exponential integrands gives the extended-real identityThe left side is finite and the first factor is finite and strictly positive. Consequently $M_R(t)$ is finite andThe same reasoning applies to $R^{\prime }$, so their moment-generating functions are finite and equal throughout $(-t_0,t_0)$. The uniqueness theorem for moment-generating functions yieldsThis proves uniqueness of a self-decomposition remainder while also checking the required finiteness; no independence between $R$ and $R^{\prime }$ is needed.

For $X\sim \mathrm{Exp}(\lambda )$, integration gives $M_X(t)=\lambda /(\lambda -t)$ for $t<\lambda$. For $0<c<1$, the candidate remainder moment-generating function isThis is the moment-generating function of the mixture distributionConstruct it by taking an exponential random variable $E$ of rate $\lambda$ and a Bernoulli random variable $J$ with $\mathbb{P}(J=1)=1-c$, with $E,J,X$ mutually independent, and setting $R_c=JE$. Then $R_c$ is independent of $X$, andon a neighborhood of zero. By the uniqueness theorem for moment-generating functions, $cX+R_c\stackrel{d}{=}X$, so the exponential distribution is a self-decomposable distribution.

For independent and identically distributed random variables $X_j\sim \mathcal{N}(0,{\sigma }^2)$, the linear combination of independent normal random variables $n^{-1/2}{\sum }_{j=1}^n{X}_j$ has expected value zero and variance $n^{-1}n{\sigma }^2={\sigma }^2$. ThusThis is exactly the requested $2$-stability. The paper's definition is a strictly stable distribution, since it allows no centering term.

First take integers $1\le k<n$. For independent copies $X_1,\dots ,X_n$ of $X$, defineThe two blocks involve disjoint families of independent random variables, so $A$ and $B$ are independent. The assumed strictly stable distribution gives $A\stackrel{d}{=}X$, $B\stackrel{d}{=}X$, andTherefore, on any fixed probability space carrying $X$ and an independent copy $X^{\prime }$, their joint probability distribution givesNow fix an arbitrary $0<c<1$ and choose rationals $r_m=k_m/n_m\in (0,1)$ tending to $c^{\alpha }$. Continuity of the real powers gives, on that fixed probability space,The left sides all have the probability distribution of $X$. Since almost sure convergence implies convergence in distribution, their limit has that same probability distribution. HenceThis proves strict stability implies self-decomposability. The limit argument uses no moment-generating function or finite moments, which is essential for general stable distributions.

A probability distribution on the countable state space $\mathcal{S}$ is precisely a row vector satisfyingA transition matrix is a stochastic matrix, meaningEvery row therefore gives the conditional probability distribution of the next state given the present state. These conditions do not require $\pi$ to be an invariant distribution of $P$.

The assumed identities are the detailed balance equations. Summing over $i$ and using the row sums of the transition matrix gives, for every $j$,All summands are nonnegative, so these countable sums are justified. Thusand $\pi$ is an invariant distribution.

For the countable-state Markov chain ergodic theorem, let the Markov chain be irreducible and positive recurrent, with its unique stationary distribution $\pi$. From any initial state, for every real $f$ satisfying ${\sum }_j{\pi }_j|f(j)|<\infty$,In particular, $n^{-1}{\sum }_{m=0}^{n-1}{1}_{\{X_m=j\}}\to {\pi }_j$ almost surely. This time-average conclusion requires no aperiodicity.

The countable-state Markov chain convergence theorem adds the hypothesis of an aperiodic Markov chain and concludesIt also follows that ${\mathbb{P}}_{\nu }(X_n=j)\to {\pi }_j$ for every initial probability distribution $\nu$. For an irreducible recurrent chain, the occupation-frequency theorem can also be stated as a limit $1/{\mathbb{E}}_j{T}_j^{+}$, interpreted as zero for a null recurrent state; in the positive recurrent case this equals ${\pi }_j$.

Under the specified evolution rule, conditional on the entire history, the next event's probability distribution depends only on the current population size. This is the time-homogeneous Markov property. Birth and immigration both increase the size by one, while death decreases it by one. Writing $q=1-p$, the transition probabilities on ${\mathbb{Z}}_{\ge 0}$ areand all other entries are zero. In particular $P(0,1)=1$ and $d_0=0$. The row sum is $[kp+\theta +kq]/(k+\theta )=1$. Every neighboring pair communicates, so this is an irreducible Markov chain, specifically a birth-death-immigration jump chain.

We use the stationary weights of a reflected birth-death chain, also verifying why they decide the question. If a stationary distribution $\pi$ exists, its equation at zero is ${\pi }_0={\pi }_1{d}_1={\pi }_0{u}_0$. At $k\ge 1$, stationarity givesThus the edge flow is constant, and its value at the boundary is zero. Every stationary distribution must therefore satisfy detailed balance on each edge. Since irreducibility forces ${\pi }_0>0$, it must be proportional toConversely, if ${\sum }_k{w}_k<\infty$, normalizing these positive weights gives a probability distribution satisfying detailed balance, hence a stationary distribution. Existence of a stationary distribution for an irreducible Markov chain on a countable state space is equivalent to positive recurrence.

The ratio isFor $p<1/2$, the limit is less than one, so the ratio test gives summability. For $p\ge 1/2$, observe that $u_k=p+\theta q/(k+\theta )>p\ge 1/2$, whereas $d_{k+1}<q\le 1/2$. Thus $w_{k+1}/w_k>1$ for every $k$, so the weights cannot be summable. This also resolves $p=1/2$ without making an inconclusive appeal to the limiting ratio. Therefore

Assume $p<1/2$, and put $a=\theta /p$ and $r=p/q\in (0,1)$. The detailed balance equations giveMultiplying from $0$ to $k-1$, the last factor telescopes, while the product involving $a$ is evaluated by the Gamma function:Consequently the requested invariant distribution can be expressed using the paper's power series asThe assumed convergence of $g_{p,\theta }(r)$ ensures that this normalization is finite and positive.

For an explicit normalizing constant, the binomial series givesThe termwise differentiation of a power series is valid for $|r|<1$, soThe last equality uses $r=p/q$ and $a=\theta /p$: $ar+\theta (1-r)=\theta /q+\theta (q-p)/q=2\theta$. ThereforeThese positive terms sum to one and satisfy the detailed balance equations, so part (a)(ii) verifies stationarity. Irreducibility makes the invariant distribution unique.

Every step changes the population by one, so $X_n$ has the parity of $n$ when $X_0=0$. Hence $\mathbb{P}(X_{2m+1}=0)=0$. A return in two steps has positive probability from every state, and all return times are even, so the period of a state in a Markov chain is exactly two.

Each transition exchanges the even and odd classes. Since $\pi$ is an invariant distribution, their masses are equal: $\pi (\text{even})=\pi (\text{odd})=1/2$. The chain sampled at times $2m$, restricted to the even states, is irreducible and positive recurrent, and it is aperiodic because its diagonal entries are positive. Its stationary distribution is $2{\pi }_k$ on even $k$. The countable-state Markov chain convergence theorem applied to this sampled chain yieldsSince ${\pi }_0>0$, the full sequence has no limit. Its time average nevertheless tends to ${\pi }_0$ by the countable-state Markov chain ergodic theorem.

Let $I_n$ count immigration events during the first $n$ transitions, and let $A_n$ count upward steps during those transitions. Write $h(k)=\theta /(k+\theta )$ for the conditional immigration probability. If $J_m$ is the indicator that transition $m$ to $m+1$ is immigration, then $J_m-h(X_m)$ is a bounded martingale difference with respect to the history that records event types as well as population sizes. The strong law for martingales with bounded increments impliesSince $h$ is bounded, the countable-state Markov chain ergodic theorem givesApplying the same argument to upward-step indicators gives $A_n/n\to {\sum }_k{\pi }_k{u}_k$ almost surely. To evaluate this denominator, sum detailed balance over all neighboring edges:Since $u_k+d_k=1$ and ${\sum }_k{\pi }_k=1$, both sums equal $1/2$. The denominator limit is positive, so division is valid and the immigration fraction among upward steps isThis is the requested answer in terms of $\pi$. Substituting part (iii) additionally gives ${\sum }_k{\pi }_k\theta /(k+\theta )=(1-r)/2$, so the fraction simplifies to $1-r=(1-2p)/(1-p)$.

Source: Oxford's released examination PDF, pages 164–167, dated 21 June 2021. Questions 1, 2 and 3 occupy bundle pages 165, 166 and 167 respectively, or pages 2, 3 and 4 of the individual paper. All three questions and every part are solved below, including all six revisions requested in 3(d). In 3(c), $c$ is understood to be an integer: the printed transition rule does not define a transition matrix for noninteger $c$.

The cumulative distribution function of the uniform distribution on $[0,1/n]$ isThe constant zero has cumulative distribution function $F(t)=1_{[0,\infty )}(t)$. Its continuity points are precisely $t\ne 0$. For fixed $t<0$, $F_n(t)=0=F(t)$; for fixed $t>0$, $F_n(t)=1=F(t)$ eventually. This proves convergence in distribution to zero directly from the definition.

Also, $\mathbb{P}(0\le U_n\le 1/n)=1$. For every $\epsilon >0$ and $n>1/\epsilon$, $\mathbb{P}(|U_n|>\epsilon )=0$. HenceThe convergence in probability conclusion holds for any realization of these random variables on a common probability space; no independence of random variables is needed.

Fix $\delta >0$. Once $|c_n-c|<\delta$, the following event inclusions hold outside $\{|X_n-X|>\delta \}$:Taking probabilities gives the explicit boundsBy the assumed convergence in probability, the error tends to zero. ThereforeThe continuity of the cumulative distribution function $F$ at $c$ lets $\delta$ decrease to zero, proving $\boxed{F_n(c_n)\to F(c).}$ This proves the needed implication directly, without invoking any theorem comparing convergence of random variables.

The definition of convergence in distribution gives $F_n(t)\to F(t)$ at each continuity point $t$ of $F$. We must handle a moving argument rather than assume this convergence is uniform.

A cumulative distribution function has at most countably many discontinuities: each is a jump of positive size, and there are at most $r$ jumps of size at least $1/r$, since their sizes sum to at most one. Thus we can choose continuity points $u_j<c<v_j$ with $u_j\uparrow c$ and $v_j\downarrow c$. For fixed $j$, eventually $u_j<c_n<v_j$. The monotonicity of each cumulative distribution function impliesFirst let $n\to \infty$, using the definition of convergence in distribution at $u_j,v_j$. Then let $j\to \infty$, using continuity at $c$:No result relating different convergence of random variables has been assumed.

Use $X_n\sim \mathrm{Unif}[0,1/n]$ from part (a). We already proved convergence in distribution to zero. For any prescribed $p\in [0,1]$, put $c_n=p/n$. Then $c_n\to 0$, and the cumulative distribution function satisfiesThis includes $p=0$ and $p=1$, since the uniform distribution assigns zero probability to each endpoint. The limit cumulative distribution function is discontinuous at zero, explaining why the conclusion of (ii) does not apply there.

Write $\Phi$ for the standard normal distribution function. The Poisson process in this part has rate one and starts at $N_0=0$.

Set $T_0=0$ and $E_j=T_j-T_{j-1}$. The Poisson interarrival times $E_j$ are independent and identically distributed random variables with the rate-one exponential distribution. In particular, $\mathbb{E}{E}_j=1$ and $\mathrm{Var}(E_j)=1$, so the finite, positive variance hypothesis of the central limit theorem holds. Since $T_n={\sum }_{j=1}^n{E}_j$,