At an almost surely finite Stopping time, a Markov process restarts from its stopped state with the same transition law and independently of the history conditional on that state.
Let switch between at rate and be killed into zero at rate . To exceed a new level it must cross it in state . Restarting there by the Strong Markov property gives the multiplicative identity , hence an exponential tail. First-step conditioning gives and . Substituting the exponential determines its rate as .
Let belong to an irreducible recurrent Discrete-time Markov chain. Write and . Define
The positive return convention is essential: itself is zero when starting at . Let count visits to before the first positive return to , including repeated holding visits at . Then
With Probability the excursion reaches . From each visit there, the Strong Markov property gives Probability of hitting before the next return to . recurrence ensures there is no third possibility of escaping both states forever. Thus, conditional on reaching , the visit count has a positive-support geometric distribution with mean .
For a two-state chain which chooses either state independently with Probability at every step, . Half of the return cycles from zero never visit one; the rest visit it a geometrically distributed number of times of mean two. The unconditional expected visit count is therefore one.

Ancestors (7)

  1. Markov chain
  2. Markov process
  3. Probability theory
  4. Probability and statistics
  5. Area of mathematics
  6. Mathematics
  7. Home

Discussion (0) New discussion

There are no discussions about this article yet.

Articles by others on the same topic (0)

There are currently no matching articles.