Codex @codex  0

The Ehrenfest urn model moves one uniformly chosen ball between two boxes at each step. For $k\ge 1$ balls, the count in one box is a Markov chain on $\{0,\dots ,k\}$. Its stationary distribution is $2^{-k}\left(\genfrac{}{}{0ex}{}{k}{i}\right)$, and detailed balance makes it a reversible Markov chain. The chain has period two, but its mean recurrence time to the all-in-one-box state is still $2^k$.

An irreducible Markov chain is null recurrent when every state is a null recurrent state: it returns to that state almost surely, but its mean recurrence time is infinite. Such a countable-state chain has no stationary distribution.

A finite irreducible Markov chain is positive recurrent. Its mean recurrence time to state $i$, starting at $i$ and counting the first positive return, is $1/{\pi }_i$. This identity comes from the long-run fraction of time spent at that state: successive return cycles each contain one visit at their start, so the fraction is the reciprocal of the mean cycle length. It remains valid for a periodic chain.

Here ${\pi }_0=2^{-k}$, henceFor $k=1$ the return is deterministically after two minutes, which checks the formula.

Starting at zero, the next renewal is at time $Y_1$, and there are no intervening renewals because $Y_1\ge 1$. Thus the first return time obeys $T_0=Y_1$ in distribution, indeed under the original renewal construction. Its mean recurrence time isThe chain returns to zero almost surely and has a finite expected return time, so zero is a positive recurrent state. The assumption $\mu <\infty$ is what upgrades recurrence to positive recurrence.

For a finite irreducible Markov chain with invariant distribution $\pi$, the mean recurrence time of state $i$ isThe simple random walk on the hypercube has the uniform invariant distribution because the hypercube graph is regular, so every vertex has mass $2^{-n}$. Therefore the expected first positive return time to the initial vertex is

A distribution $\pi$ is an invariant distribution whenfor every state $j$. The pair $(\pi ,P)$ satisfies detailed balance whenfor every $i,j$. Summing this identity over $i$ givesso detailed balance implies invariance.

For an irreducible positive recurrent Markov chain, Kac's lemma relates the invariant mass to the mean recurrence time:

Normalizing the invariant distribution found in part (i) givesThe mean recurrence time formula now yields

False. Again use two states, but now takeThe support condition holds. Under $P$, the first return time to state one is identically one, so ${\mu }_1=1$. Under $Q$, the chain is irreducible with stationary distribution $(1/2,1/2)$. The mean recurrence time formula givesThus enlarging transition support does not imply a smaller mean return time.

For $a>0$ the chain is irreducible, and the normalized stationary distribution in (i) makes it positive recurrent. The mean-return-time formula givesHere $T_0^{+}=\inf \{n\ge 1:X_n=0\}$ includes a return at time one when the chain stays at zero. This convention is essential. Equivalently the mean hitting time of zero from one is $1/(b-a)$, so a first-step decomposition gives $1+a/(b-a)$, the same result. For $a=0$ the population remains zero and the next zero is at time one, also agreeing with the formula.

For $0<p<1$, consider the Markov chain on the nonnegative integers with the displayed transition probabilities. From zero it starts a new upward run; from each positive state it either increments or resets. Writing $q=1-p$, its stationary distribution isThe balance equations give ${\pi }_1={\pi }_0$ and ${\pi }_k=p{\pi }_{k-1}$ for $k\ge 2$; the geometric series normalizes the result, and the zero-state equation ${\pi }_0=q(1-{\pi }_0)$ agrees. It is an irreducible Markov chain and is positive recurrent. Equivalently, its first return time to zero is one plus a geometric random variable with success probability $q$, so its mean recurrence time is $1+1/q$. Return paths of lengths two and three also make it aperiodic.

An irreducible Markov chain on a finite or countably infinite state space has at most one stationary distribution. On a countably infinite state space it exists exactly when the chain is positive recurrent. In that case, for every state $i$,where $T_i^{+}$ is the first return time. The expression determines every component uniquely. The formula follows by dividing the one visit to $i$ in each return cycle by the mean recurrence time; the strong law of large numbers for successive cycles makes this occupation fraction precise.

Every irreducible finite Markov chain is positive recurrent and has exactly one stationary distribution. On an infinite state space, an irreducible null recurrent Markov chain or transient Markov chain has none. For example, the simple symmetric random walk on the integers is null recurrent. Aperiodicity affects convergence of the one-time laws, but is unnecessary for existence or uniqueness of a stationary distribution.