Past exam of the probability course of the University of Oxford / 2023 / 3

A probability distribution on the countable state space $\mathcal{S}$ is precisely a row vector satisfyingA transition matrix is a stochastic matrix, meaningEvery row therefore gives the conditional probability distribution of the next state given the present state. These conditions do not require $\pi$ to be an invariant distribution of $P$.

The assumed identities are the detailed balance equations. Summing over $i$ and using the row sums of the transition matrix gives, for every $j$,All summands are nonnegative, so these countable sums are justified. Thusand $\pi$ is an invariant distribution.

For the countable-state Markov chain ergodic theorem, let the Markov chain be irreducible and positive recurrent, with its unique stationary distribution $\pi$. From any initial state, for every real $f$ satisfying ${\sum }_j{\pi }_j|f(j)|<\infty$,In particular, $n^{-1}{\sum }_{m=0}^{n-1}{1}_{\{X_m=j\}}\to {\pi }_j$ almost surely. This time-average conclusion requires no aperiodicity.

The countable-state Markov chain convergence theorem adds the hypothesis of an aperiodic Markov chain and concludesIt also follows that ${\mathbb{P}}_{\nu }(X_n=j)\to {\pi }_j$ for every initial probability distribution $\nu$. For an irreducible recurrent chain, the occupation-frequency theorem can also be stated as a limit $1/{\mathbb{E}}_j{T}_j^{+}$, interpreted as zero for a null recurrent state; in the positive recurrent case this equals ${\pi }_j$.

Under the specified evolution rule, conditional on the entire history, the next event's probability distribution depends only on the current population size. This is the time-homogeneous Markov property. Birth and immigration both increase the size by one, while death decreases it by one. Writing $q=1-p$, the transition probabilities on ${\mathbb{Z}}_{\ge 0}$ areand all other entries are zero. In particular $P(0,1)=1$ and $d_0=0$. The row sum is $[kp+\theta +kq]/(k+\theta )=1$. Every neighboring pair communicates, so this is an irreducible Markov chain, specifically a birth-death-immigration jump chain.

We use the stationary weights of a reflected birth-death chain, also verifying why they decide the question. If a stationary distribution $\pi$ exists, its equation at zero is ${\pi }_0={\pi }_1{d}_1={\pi }_0{u}_0$. At $k\ge 1$, stationarity givesThus the edge flow is constant, and its value at the boundary is zero. Every stationary distribution must therefore satisfy detailed balance on each edge. Since irreducibility forces ${\pi }_0>0$, it must be proportional toConversely, if ${\sum }_k{w}_k<\infty$, normalizing these positive weights gives a probability distribution satisfying detailed balance, hence a stationary distribution. Existence of a stationary distribution for an irreducible Markov chain on a countable state space is equivalent to positive recurrence.

The ratio isFor $p<1/2$, the limit is less than one, so the ratio test gives summability. For $p\ge 1/2$, observe that $u_k=p+\theta q/(k+\theta )>p\ge 1/2$, whereas $d_{k+1}<q\le 1/2$. Thus $w_{k+1}/w_k>1$ for every $k$, so the weights cannot be summable. This also resolves $p=1/2$ without making an inconclusive appeal to the limiting ratio. Therefore

Assume $p<1/2$, and put $a=\theta /p$ and $r=p/q\in (0,1)$. The detailed balance equations giveMultiplying from $0$ to $k-1$, the last factor telescopes, while the product involving $a$ is evaluated by the Gamma function:Consequently the requested invariant distribution can be expressed using the paper's power series asThe assumed convergence of $g_{p,\theta }(r)$ ensures that this normalization is finite and positive.

For an explicit normalizing constant, the binomial series givesThe termwise differentiation of a power series is valid for $|r|<1$, soThe last equality uses $r=p/q$ and $a=\theta /p$: $ar+\theta (1-r)=\theta /q+\theta (q-p)/q=2\theta$. ThereforeThese positive terms sum to one and satisfy the detailed balance equations, so part (a)(ii) verifies stationarity. Irreducibility makes the invariant distribution unique.

Every step changes the population by one, so $X_n$ has the parity of $n$ when $X_0=0$. Hence $\mathbb{P}(X_{2m+1}=0)=0$. A return in two steps has positive probability from every state, and all return times are even, so the period of a state in a Markov chain is exactly two.

Each transition exchanges the even and odd classes. Since $\pi$ is an invariant distribution, their masses are equal: $\pi (\text{even})=\pi (\text{odd})=1/2$. The chain sampled at times $2m$, restricted to the even states, is irreducible and positive recurrent, and it is aperiodic because its diagonal entries are positive. Its stationary distribution is $2{\pi }_k$ on even $k$. The countable-state Markov chain convergence theorem applied to this sampled chain yieldsSince ${\pi }_0>0$, the full sequence has no limit. Its time average nevertheless tends to ${\pi }_0$ by the countable-state Markov chain ergodic theorem.

Let $I_n$ count immigration events during the first $n$ transitions, and let $A_n$ count upward steps during those transitions. Write $h(k)=\theta /(k+\theta )$ for the conditional immigration probability. If $J_m$ is the indicator that transition $m$ to $m+1$ is immigration, then $J_m-h(X_m)$ is a bounded martingale difference with respect to the history that records event types as well as population sizes. The strong law for martingales with bounded increments impliesSince $h$ is bounded, the countable-state Markov chain ergodic theorem givesApplying the same argument to upward-step indicators gives $A_n/n\to {\sum }_k{\pi }_k{u}_k$ almost surely. To evaluate this denominator, sum detailed balance over all neighboring edges:Since $u_k+d_k=1$ and ${\sum }_k{\pi }_k=1$, both sums equal $1/2$. The denominator limit is positive, so division is valid and the immigration fraction among upward steps isThis is the requested answer in terms of $\pi$. Substituting part (iii) additionally gives ${\sum }_k{\pi }_k\theta /(k+\theta )=(1-r)/2$, so the fraction simplifies to $1-r=(1-2p)/(1-p)$.