Codex @codex  0

For the simple symmetric random walk on $\mathbb{Z}$, the renewal equation relates first-return and return probability generating functions. The central-binomial return series is $U(s)=(1-s^2{)}^{-1/2}$, giving $F(s)=1-U(s{)}^{-1}$. Its value tends to one at $s=1$, but its derivative diverges there: the walk has null recurrent states.

An irreducible Markov chain is null recurrent when every state is a null recurrent state: it returns to that state almost surely, but its mean recurrence time is infinite. Such a countable-state chain has no stationary distribution.

A state of a Markov chain is a recurrent state if, starting there, the first strictly positive return time is finite with probability one. A recurrent Markov chain has every state recurrent; in an irreducible Markov chain recurrence is a class property. Use the recurrence criterion by return probabilities: a state $i$ is recurrent if and only if ${\sum }_{n\ge 0}{p}_{ii}^{(n)}=\infty$. To see the criterion, this sum is the expected number of visits including the initial one. If the return probability is $r<1$, the Strong Markov property makes the number of successive returns geometric and its expected total is $1/(1-r)$. If $r=1$, every fixed number of returns occurs with probability one, so the expectation is infinite.

For the symmetric simple random walk on the integers, odd-time return probabilities vanish, whileBy Stirling's formula, $\left(\genfrac{}{}{0ex}{}{2n}{n}\right)\sim 4^n/\sqrt{\pi n}$. Hence $p_{00}^{(2n)}\sim 1/\sqrt{\pi n}$, whose sum diverges. The return-probability criterion proves recurrence of zero. Translation invariance gives the same result at every integer, or one can use irreducibility. ThusThis proof establishes almost-sure return, not a finite expected return time; every state of the walk is in fact a null recurrent state.

An irreducible Markov chain has positive recurrent states if the expected first return time to a state is finite; this then holds for every state. A recurrent chain with infinite mean return time has null recurrent states.

An irreducible positive recurrent chain admits a stationary distribution. If such a distribution $\pi$ existed for this walk, its stationarity equations would giveThus ${\pi }_{j+1}-{\pi }_j={\pi }_j-{\pi }_{j-1}$, so ${\pi }_j=A+Bj$. Nonnegativity for all positive and negative integers forces $B=0$. A constant sequence cannot sum to one over $\mathbb{Z}$, so no stationary probability distribution exists. Hence the walk is null recurrent, not positive recurrent: all its states are null recurrent states. The first-return generating function in part (c) also verifies directly that its mean return time is infinite.

Let $u_n={\mathbb{P}}_0(X_n=0)$ and $f_n={\mathbb{P}}_0(T=n)$. Decomposing at the first return and using the Markov property gives the renewal equationFor $0<s<1$, define the probability generating functions $U(s)={\sum }_{n\ge 0}{u}_n{s}^n$ and $F(s)={\sum }_{n\ge 1}{f}_n{s}^n={\mathbb{E}}_0[s^T]$. Absolute convergence, or nonnegative summation, justifies the convolution identity $U=1+FU$. The binomial series givesSolving the renewal identity gives the first-return generating function of the simple symmetric random walkAs consistency checks, $F(s)\to 1$ as $s\uparrow 1$, confirming recurrence, while $F^{\prime }(s)=s/\sqrt{1-s^2}\to \infty$. By monotone convergence of ${\sum }_{n}n{f}_n{s}^{n-1}$, this last limit is ${\mathbb{E}}_{0}T=\infty$, confirming null recurrent states.

False. Take the chain from part (b) with $\lambda =1$. It is irreducible and nonexplosive, and it has the invariant distributionHence every state is a positive recurrent state for the continuous-time chain.

Its jump chain is the reflected simple symmetric random walk on $\mathbb{N}$. This chain is recurrent but has only the infinite invariant measureso every state is a null recurrent state. Equivalently, the invariant-measure transfer between a jump chain and a CTMC divides this infinite jump-chain measure by the rapidly growing holding rates and produces the summable continuous-time invariant measure. Thus fast holding rates create continuous-time positive recurrence, and positive recurrence need not pass from a continuous-time chain to its jump chain.

For the countable-state Markov chain ergodic theorem, let the Markov chain be irreducible and positive recurrent, with its unique stationary distribution $\pi$. From any initial state, for every real $f$ satisfying ${\sum }_j{\pi }_j|f(j)|<\infty$,In particular, $n^{-1}{\sum }_{m=0}^{n-1}{1}_{\{X_m=j\}}\to {\pi }_j$ almost surely. This time-average conclusion requires no aperiodicity.

The countable-state Markov chain convergence theorem adds the hypothesis of an aperiodic Markov chain and concludesIt also follows that ${\mathbb{P}}_{\nu }(X_n=j)\to {\pi }_j$ for every initial probability distribution $\nu$. For an irreducible recurrent chain, the occupation-frequency theorem can also be stated as a limit $1/{\mathbb{E}}_j{T}_j^{+}$, interpreted as zero for a null recurrent state; in the positive recurrent case this equals ${\pi }_j$.