Past exam of the mathematics course of the University of Cambridge / 2008 / ii / Paper 2 / 26I (Applied Probability)

The arrows in the PDF give $C_i\to C_{i+1}$ at rate $\lambda$, $W_i\to W_{i-1}$ at rate $\mu$ for $i\ge 1$, and switches $C_i\to W_i$ and $W_i\to C_i$ at rates $\alpha$ and $\beta$. All rates are positive. One can travel upwards in level in the $C$ phase and downwards in the $W$ phase, switching at either end. Hence any state reaches any other through a finite positive-rate path: the continuous-time Markov chain is irreducible. Its total jump rates are uniformly bounded, so it is also nonexplosive.

Condition on the first jump, putting $h_{W_0}=1$. The hitting probabilities obeyThe relevant characterization is that a hitting probability is the minimal nonnegative harmonic extension of its target-boundary value. Thus among nonnegative solutions of these equations with $h_{W_0}=1$, choose the componentwise least one. Other harmonic solutions need not be probabilities. This also identifies the solution without imposing an unjustified boundary condition at infinity.

Use the $C$ equation at level $i-1$ to express $h_{C_i}$ in terms of the preceding level, then substitute it into the $W$ equation at level $i$. This yieldsThe standard eigenpair is $\boxed{1,(1,1{)}^T}$. Each row sums to one because constant functions are harmonic for every Markov generator. This recurrence matrix need not have nonnegative entries; it encodes a harmonic equation, rather than a transition matrix. The superscript $(i)$ labels the level vector and is not an additional printed subpart.

Since one eigenvalue is one, the other is the determinant. Direct calculation givesIndeed the first row requires $(1+\alpha /\lambda -\vartheta )b=\alpha /\lambda$, and the second row then follows from the characteristic equation. At $\vartheta =1$, $b=1$ and the eigenvectors coincide; the nonidentity matrix has a repeated eigenvalue and a Jordan block.

If $\vartheta >1$, the general level solution is a constant multiple of $(1,1{)}^T$ plus a multiple of ${\vartheta }^i(b,1{)}^T$. Boundedness of probabilities removes the growing component, and $h_{W_0}=1$ then fixes the constant to one. If $\vartheta =1$, the nonzero Jordan part produces linear growth unless the initial vector lies in the one-dimensional eigenspace; the same conclusion follows. ThusStarting at $W_0$, the chain leaves to $C_0$ and returns with probability one. An irreducible chain with a recurrent state has all states recurrent, so the chain is recurrent in both cases.

Here $\vartheta <1$ is equivalent to $\mu \alpha <\lambda \beta$, which gives $b<1$ from its explicit formula. Any nonnegative solution is of the formsince its second component at level zero must be one. Letting $i\to \infty$ forces $a\ge 0$ for nonnegativity. The choice $a=0$ is nonnegative, and subtracting it from any other such solution gives $a[(1,1{)}^T-{\vartheta }^i(b,1{)}^T]\ge 0$. Therefore minimality selectsIn particular the return probability after leaving $W_0$ is $b<1$, so the irreducible chain is transient. These formulas are the hitting probabilities for a velocity-switching Markov ladder.

Combining the eigenvalue cases givesThis agrees with the interior drift: the switching phase spends proportions $\beta /(\alpha +\beta )$ and $\alpha /(\alpha +\beta )$ in $C$ and $W$, giving level drift $(\lambda \beta -\mu \alpha )/(\alpha +\beta )$. The harmonic calculation establishes recurrence even at the zero-drift boundary, rather than inferring it from drift alone.