Past exam of the probability course of the University of Oxford / 2021 / 3

Put $q=1-p=a/(a+b)$. The random walk has independent and identically distributed increments $Y_n$ equal to $a$ with probability $p=b/(a+b)$ and $-b$ with probability $q$. Thus $S_0=0$, $S_n={\sum }_{j=1}^n{Y}_j$, and $\mathbb{E}{Y}_n=0$.

The history $\sigma (S_0,\dots ,S_n)$ is contained in $\sigma (Y_1,\dots ,Y_n)$. The increment $Y_{n+1}$ is independent of this history by the given independence of random variables. ThereforeThis depends only on the current state and not on time. It is precisely the time-homogeneous Markov property, so $S_n$ is a Markov chain with transition probabilitiesThe conditional identity is an identity that holds almost surely; histories of zero probability require no division by their probabilities.

Here $a,b$ are coprime integers.

Fix any two integers $i,j$. By Bézout's identity, there are integers $u,v$ with $au-bv=j-i$. Replacing $(u,v)$ by $(u+kb,v+ka)$ leaves this identity unchanged. Choose $k$ large enough that both entries are nonnegative. A path with $u+kb$ upward steps and $v+ka$ downward steps then takes $i$ to $j$, with strictly positive probability $p^{u+kb}{q}^{v+ka}$. There are no boundary restrictions on this state space.

This applies in both directions, so every pair of states communicates. Hence

A return path with $u$ upward steps and $v$ downward steps satisfies $au=bv$. Since $a,b$ are coprime, this is equivalent to $u=bk$ and $v=ak$ for an integer $k\ge 1$. Its length is therefore $k(a+b)$.

Conversely, for each such $k$, any ordering of $bk$ upward and $ak$ downward steps is an allowed positive-probability return path. The possible strictly positive return times are exactly the positive multiples of $a+b$. By the definition of the period of a state in a Markov chain as their greatest common divisor,In particular, none of the states is aperiodic.

Suppose a stationary distribution $\pi$ exists. Its stationarity equations areSince ${\sum }_j{\pi }_j=1$, its masses tend to zero as $|j|\to \infty$, and a positive maximum $M={\pi }_{j_0}$ is attained. Both terms on the right are at most $M$. Since $p,q>0$ and $p+q=1$, equality forces ${\pi }_{j_0-a}={\pi }_{j_0+b}=M$. Applying the same argument repeatedly at $j_0-a,j_0-2a,\dots$ givesThese are infinitely many distinct states, contradicting ${\sum }_j{\pi }_j=1$. ThusThis maximum principle proof uses neither a finite expected value of the state nor the coprimality assumption.

First identify the recurrence type, rather than infer an occupation limit from a one-time probability. At time $k(a+b)$, return to zero requires exactly $kb$ upward steps. The binomial distribution giveswhere the last step is Stirling's approximation. At all other positive times the return probability is zero. Hence ${\sum }_{n\ge 0}{\mathbb{P}}_0(S_n=0)=\infty$. The recurrence criterion by return probabilities proves the recurrent property; by the irreducible Markov chain property established in (i), every state is recurrent. The equivalence of positive recurrence and existence of a stationary distribution for an irreducible Markov chain on a countable state space, together with (iii), makes this chain null recurrent.

For completeness, the zero occupation proportion follows directly by treating returns as a renewal process. Let $L_1,L_2,\dots$ be successive strictly positive return-cycle lengths to zero. The Strong Markov property makes them independent and identically distributed, the recurrent property makes them finite almost surely, and the null recurrent property gives $\mathbb{E}{L}_1=\infty$. For each positive integer $K$, the strong law of large numbers applied to $\min (L_j,K)$ givesTake a common event of probability one for these countably many $K$. By the monotone convergence theorem, the right side increases to infinity. Thus the number of completed return cycles by time $n$, divided by $n$, tends to zero. Counting the initial visit changes this ratio by at most $1/n$. Consequently,The walk visits zero infinitely often, but the asymptotic fraction of time spent there is zero.

Take an integer $c\ge \max \{a,b\}$. On $I=\{-c,\dots ,c\}$, the modified transition matrix $Q=(q_{i,j})$ agrees with $P$. Above $c$, it stays put with probability $a/(a+b)$ and jumps to each of $c-b+1,\dots ,c$ with probability $1/(a+b)$. Below $-c$, it stays put with probability $b/(a+b)$ and jumps to each of $-c,\dots ,-c+a-1$ with probability $1/(a+b)$. All unspecified entries are zero. These rows sum to one. The integer qualification is necessary: if $c$ were nonintegral, $i=\lfloor c\rfloor +1$ would fit none of the printed row conditions.

DefineAn interior jump cannot go below $-c-b$ or above $c+a$, and every exterior reset lands in $I$. Thus $C$ is closed.

To prove that it is one communicating class, start anywhere in $C$. From $L$ or $U$, a reset can reach $I$ with positive probability. From any state of $I$, repeated $+a$ steps reach some state of $U$, after staying in $I$ up to the last step. A reset from this state can reach every point of the consecutive blockEvery integer $j\in \{-c-b,\dots ,c\}$ has the form $t-kb$ for some $t\in B$ and an integer $k\ge 0$. Repeated $-b$ steps from $t$ reach $j$, with all departure states in $I$ until a possible last step into $L$. This gives access to all of $I\cup L$. Finally, each $j\in U$ can be reached from $j-a\in I$. Every path described has strictly positive probability. Therefore every state in $C$ reaches every other state in $C$.

Each state outside $C$ can only stay put or reset into $I$, and no transition from $C$ returns to it. Different states outside $C$ cannot reach one another. Hence the complete list of communicating classes isThe finite class $C$ is the unique closed communicating class. The exterior singleton classes are transient communicating classes, since their self-transition probabilities are strictly less than one. Notice that this reachability proof does not require $a,b$ to be coprime.

The restriction $Q_C$ is a finite Markov chain that is an irreducible Markov chain. It is aperiodic, because its upper states have self-transition probability $a/(a+b)>0$, and the period is constant on a communicating class. We now find its stationary distribution.

Every column of $Q_C$ sums to one. Here is the full incoming-mass check, with each total measured in units of $1/(a+b)$. For $j\in L$, the incoming contributions are $a$ from $j+b\in I$ and $b$ from the self-transition. For $j\in U$, they are $b$ from $j-a\in I$ and $a$ from the self-transition.

For $j\in I$, the upward predecessor $j-a$ lies in $I$ exactly when $j\ge -c+a$, contributing $b$ units. When $j\le -c+a-1$, that predecessor is replaced by resets from all $b$ states of $L$, again contributing $b$ units. Independently, the downward predecessor $j+b$ contributes $a$ units when $j\le c-b$; when $j\ge c-b+1$, resets from the $a$ states of $U$ contribute those $a$ units instead. Thus every column total is $a+b$ units.

Therefore $Q_C$ is a doubly stochastic matrix, and the uniform distribution on $C$ is stationary:The countable-state Markov chain convergence theorem, applied to the finite closed class $C$, gives $Q_C^n(i,j)\to {\pi }_j$ for every $i\in C$. Its hypotheses hold: $C$ is finite and irreducible, hence positive recurrent, and it is aperiodic.

If $i_0>c+a$, the hitting time $H$ of $C$ has $\mathbb{P}(H>n)=(a/(a+b){)}^n$. If $i_0<-c-b$, the corresponding tail is $(b/(a+b){)}^n$. Thus $H<\infty$ almost surely from every exterior initial state. Conditioning on $H$ and $R_H$, use convergence within $C$ after entrance and the bounded convergence theorem; equivalently, first restrict to $H\le K$ and then let $K\to \infty$. This proves, for every $i_0\in \mathbb{Z}$,The limiting distribution is independent of $i_0$.

Let $d=\gcd (a,b)>1$ and put $a^{\prime }=a/d$, $b^{\prime }=b/d$. Then $a^{\prime },b^{\prime }$ are coprime integers, and $p=b^{\prime }/(a^{\prime }+b^{\prime })$. The six requested revisions are as follows.

(b)(i): communicating classes. Every increment is divisible by $d$, so the residue class modulo $d$ is preserved. Conversely, if $j-i$ is divisible by $d$, Bézout's identity applied to $a^{\prime },b^{\prime }$ gives a path just as in (b)(i). Thus the original Markov chain is reducible on $\mathbb{Z}$, with exactly the $d$ closed communicating classesThe specified walk starting at zero stays in $d\mathbb{Z}$.

(b)(ii): period of a state in a Markov chain. The return equation $au=bv$ reduces to $a^{\prime }u=b^{\prime }v$. Hence $u=b^{\prime }k$, $v=a^{\prime }k$. For every state, the period of a state in a Markov chain is thereforeThis is still at least two, so no state is aperiodic.

(b)(iii): stationary distribution. The conclusion is unchanged: there is no stationary distribution on $\mathbb{Z}$. The maximum-mass proof in (b)(iii) applies verbatim, without using coprimality. Equivalently, each closed communicating class, after translating and dividing its state labels by $d$, is the same centered random walk with step sizes $a^{\prime },-b^{\prime }$ and has no stationary distribution.

(b)(iv): occupation of zero. On $d\mathbb{Z}$, the rescaled walk $S_n/d$ has exactly the transition probabilities of the coprime walk with parameters $a^{\prime },b^{\prime }$. The return-probability argument in (b)(iv) therefore makes this class null recurrent. The renewal process proof of the occupation limit applies unchanged:Starting in another residue class would make every visit to zero impossible, also giving occupation proportion zero.

(c)(i): modified communicating classes. These are unchanged: $C=\{-c-b,\dots ,c+a\}$ is one finite closed communicating class, and all other states are transient singleton classes. The original proof used consecutive reset destinations and never used coprimality. Resetting to the $b$ consecutive integers $c-b+1,\dots ,c$ connects different residue classes, so it would be incorrect to split $C$ according to residue classes modulo $d$.

(c)(ii): limiting distribution. The column-sum calculation, the irreducible Markov chain property on a finite class, positive self-transition probabilities, and geometric entrance argument also required no coprimality. Thus for every initial state the same convergence in distribution holds:In particular, the denominator $2c+a+b+1$ is unchanged; it is not divided by $d$.