A distribution

π

π_{j} = \sum_{i \in I} π_{i} p_{ij}

for every state

j

. The pair

(π, P)

π_{i} p_{ij} = π_{j} p_{ji}

for every

i, j

. Summing this identity over

i

gives

\sum_{i} π_{i} p_{ij} = \sum_{i} π_{j} p_{ji} = π_{j} \sum_{i} p_{ji} = π_{j},

so detailed balance implies invariance.

For an irreducible positive recurrent Markov chain, Kac's lemma relates the invariant mass to the mean recurrence time:

E_{i} T_{i}^{+} = \frac{1}{π _{i}} .

ii

Solution

The renewal-reward form of Kac's lemma says that the expected number of visits to state

i

during one return cycle from

k

k

\frac{π _{i}}{π _{k}} .

Here time spent means the number of discrete time instants at which the chain occupies

i

between consecutive visits to

k

There are no discussions about this article yet.

There are currently no matching articles.