Let belong to an irreducible recurrent Discrete-time Markov chain. Write and . Define
The positive return convention is essential: itself is zero when starting at . Let count visits to before the first positive return to , including repeated holding visits at . Then
With Probability the excursion reaches . From each visit there, the Strong Markov property gives Probability of hitting before the next return to . recurrence ensures there is no third possibility of escaping both states forever. Thus, conditional on reaching , the visit count has a positive-support geometric distribution with mean .
For a two-state chain which chooses either state independently with Probability at every step, . Half of the return cycles from zero never visit one; the rest visit it a geometrically distributed number of times of mean two. The unconditional expected visit count is therefore one.

Articles by others on the same topic (0)

There are currently no matching articles.