7H (Markov Chains)

Let

q = P_{i} (T_{i} < \infty)

. By the Strong Markov property, after every return to

i

the chance of another return is again

q

. Hence

P_{i} (V_{i} = k) = (1 - q) q^{k - 1}, k = 1, 2, \dots .

Thus $V_{i}$ is geometric with success parameter $1 - q$ ; when $q = 1$ , it is infinite almost surely.

b

In a transient irreducible chain, every state

j

is transient, so the expected number of visits satisfies

\sum_{n = 0}^{\infty} P^{n} (i, j) = E_{i} V_{j} < \infty.

The terms are nonnegative, and every convergent series has terms tending to zero. Therefore

P^{n} (i, j) \to 0 .

There are no discussions about this article yet.

There are currently no matching articles.