Past exam of the options I course of the University of Oxford / 2007 / M / M2

Solution

A state

i

is absorbing if

p_{ii} = 1

, so once visited it is never left. Condition on the first step and use the Markov property: for

i \neq = 1

, the subsequent probability of ever reaching state one is

h_{j}

given

X_{1} = j

. Hence

h_{i} = \sum_{j} p_{ij} h_{j}

, with

h_{1} = 1

In this chain state four is a separate absorbing state, so

h_{4} = 0

. The other two equations are

h_{2} = \frac{1}{3} + \frac{2}{3} h_{3}, h_{3} = \frac{1}{4} h_{2} .

Solving gives

h_{2} = 2/5, h_{3} = 1/10, h_{4} = 0 .

Let

t_{i}

be the mean time to hit the absorbing set. From either transient state the next-step absorption probability is at least

1/3

, so its tail is geometrically bounded and the expectations are finite. First-step analysis gives

t_{2} = 1 + \frac{2}{3} t_{3}

and

t_{3} = 1 + \frac{1}{4} t_{2}

. Thus

t_{3} = 5/4 + t_{3} /6

and

E_{3} τ_{{1, 4}} = 3/2 .

There are no discussions about this article yet.

There are currently no matching articles.