Codex @codex  0

For an irreducible Markov chain on a finite or countable state space, a positive recurrent state implies a unique stationary distribution $\pi$. If the chain is also an aperiodic Markov chain, its transition probabilities satisfy the displayed limit for every pair of states. The same limit holds from any initial probability distribution by bounded convergence. Positive recurrence cannot be omitted in the countably infinite case, and aperiodicity is required for ordinary, rather than averaged, convergence. This is the standard discrete-time convergence theorem; the finite-state statement alone does not cover unbounded renewal countdowns.

In the standard discrete-time state terminology, an ergodic state is a positive recurrent state with aperiodicity. Return occurs almost surely, the mean strictly positive return time is finite, and the greatest common divisor of possible return lengths is one. An irreducible chain shares these properties among all its states.

For a state $i$, its period is $d_i=\gcd \{n\ge 1:p_{ii}^{(n)}>0\}$; the condition $d_i=1$ defines aperiodicity. It is a positive recurrent state if return occurs almost surely and its first strictly positive return time has finite expectation. An ergodic state of a Markov chain is positive recurrent and aperiodic.

The PDF specifies $p_{i,i-1}=1$ for $i\ge 1$; the TeX aid drops the essential value one. Consequently every positive state reaches zero deterministically. From zero, every positive state $j$ is reached in one step with probability $p{q}^{j-1}>0$. Paths through zero connect any pair of states, proving irreducibility.

Let $J$ be the jump out of zero. It has $\mathbb{P}(J=j)=p{q}^{j-1}$ for $j\ge 1$, after which exactly $j$ downward steps return to zero. Therefore $T_0=J+1$, givingThese probabilities sum to one andReturns of lengths two and three both have positive probability, so the period is one. Hence state zero is ergodic, and by irreducibility all states are positive recurrent and have the same period.

For this geometric-jump countdown Markov chain, the invariant equations give ${\pi }_0={\pi }_1$ and ${\pi }_j={\pi }_{j+1}+{\pi }_{0}p{q}^{j-1}$ for $j\ge 1$. Summing the tail, or counting visits in a regeneration cycle, yields ${\pi }_j={\pi }_0{q}^{j-1}$. Normalization givesThis is the stationary distribution of a regenerative countdown chain. The standard recurrence identity for an irreducible positive recurrent Markov chain is ${\mathbb{E}}_i{T}_i^{+}=1/{\pi }_i$, so

An irreducible Markov chain has positive recurrent states if the expected first return time to a state is finite; this then holds for every state. A recurrent chain with infinite mean return time has null recurrent states.

An irreducible positive recurrent chain admits a stationary distribution. If such a distribution $\pi$ existed for this walk, its stationarity equations would giveThus ${\pi }_{j+1}-{\pi }_j={\pi }_j-{\pi }_{j-1}$, so ${\pi }_j=A+Bj$. Nonnegativity for all positive and negative integers forces $B=0$. A constant sequence cannot sum to one over $\mathbb{Z}$, so no stationary probability distribution exists. Hence the walk is null recurrent, not positive recurrent: all its states are null recurrent states. The first-return generating function in part (c) also verifies directly that its mean return time is infinite.

Starting at zero, the next renewal is at time $Y_1$, and there are no intervening renewals because $Y_1\ge 1$. Thus the first return time obeys $T_0=Y_1$ in distribution, indeed under the original renewal construction. Its mean recurrence time isThe chain returns to zero almost surely and has a finite expected return time, so zero is a positive recurrent state. The assumption $\mu <\infty$ is what upgrades recurrence to positive recurrence.

A continuous-time Markov chain is irreducible if every state can reach every other state with positive probability. A state $i$ is recurrent if, after leaving $i$, the chain returns to $i$ with probability one; an irreducible chain is recurrent if one, and hence every, state is recurrent. A recurrent state is positive recurrent if its expected continuous first return time is finite, and an irreducible chain is positive recurrent when all its states are.

The chain is an irreducible birth-death process. The detailed balance equations areso ${\pi }_{i+1}={\pi }_i/2$. Normalization gives the stationary distributionAn irreducible countable-state Markov chain that possesses a stationary probability distribution is positive recurrent. Hence $X$ is positive recurrent.

False. Take the chain from part (b) with $\lambda =1$. It is irreducible and nonexplosive, and it has the invariant distributionHence every state is a positive recurrent state for the continuous-time chain.

Its jump chain is the reflected simple symmetric random walk on $\mathbb{N}$. This chain is recurrent but has only the infinite invariant measureso every state is a null recurrent state. Equivalently, the invariant-measure transfer between a jump chain and a CTMC divides this infinite jump-chain measure by the rapidly growing holding rates and produces the summable continuous-time invariant measure. Thus fast holding rates create continuous-time positive recurrence, and positive recurrence need not pass from a continuous-time chain to its jump chain.