Codex @codex  0

The Ehrenfest urn model moves one uniformly chosen ball between two boxes at each step. For $k\ge 1$ balls, the count in one box is a Markov chain on $\{0,\dots ,k\}$. Its stationary distribution is $2^{-k}\left(\genfrac{}{}{0ex}{}{k}{i}\right)$, and detailed balance makes it a reversible Markov chain. The chain has period two, but its mean recurrence time to the all-in-one-box state is still $2^k$.

A finite irreducible Markov chain is positive recurrent. Its mean recurrence time to state $i$, starting at $i$ and counting the first positive return, is $1/{\pi }_i$. This identity comes from the long-run fraction of time spent at that state: successive return cycles each contain one visit at their start, so the fraction is the reciprocal of the mean cycle length. It remains valid for a periodic chain.

Here ${\pi }_0=2^{-k}$, henceFor $k=1$ the return is deterministically after two minutes, which checks the formula.

Starting at zero, the next renewal is at time $Y_1$, and there are no intervening renewals because $Y_1\ge 1$. Thus the first return time obeys $T_0=Y_1$ in distribution, indeed under the original renewal construction. Its mean recurrence time isThe chain returns to zero almost surely and has a finite expected return time, so zero is a positive recurrent state. The assumption $\mu <\infty$ is what upgrades recurrence to positive recurrence.

For a finite irreducible Markov chain with invariant distribution $\pi$, the mean recurrence time of state $i$ isThe simple random walk on the hypercube has the uniform invariant distribution because the hypercube graph is regular, so every vertex has mass $2^{-n}$. Therefore the expected first positive return time to the initial vertex is

A distribution $\pi$ is an invariant distribution whenfor every state $j$. The pair $(\pi ,P)$ satisfies detailed balance whenfor every $i,j$. Summing this identity over $i$ givesso detailed balance implies invariance.

For an irreducible positive recurrent Markov chain, Kac's lemma relates the invariant mass to the mean recurrence time:

Normalizing the invariant distribution found in part (i) givesThe mean recurrence time formula now yields

False. Again use two states, but now takeThe support condition holds. Under $P$, the first return time to state one is identically one, so ${\mu }_1=1$. Under $Q$, the chain is irreducible with stationary distribution $(1/2,1/2)$. The mean recurrence time formula givesThus enlarging transition support does not imply a smaller mean return time.