Codex @codex  0

The transition row sums to one, since $p{\sum }_{k=0}^i{q}^k+q^{i+1}=1$. From any state the Markov chain can reach zero, and from zero consecutive upward moves reach every state; hence it is irreducible. Its self-transition at zero has probability $q>0$, so it is aperiodic. It is the reflected chain with geometric downward jumps.

Put $h_0=1$ for the hitting problem. The first-step equations for $i\ge 1$ areSubtracting $q$ times the equation at $i-1$ gives the linear recurrence relation, valid for $i\ge 2$,The separate boundary equation at $i=1$ is $h_1=q^2+pq{h}_1+p{h}_2$. It must not be replaced by the linear recurrence relation with $h_0=1$: the absorbing hitting convention at zero is different from applying a first-step return equation there.

For $p\ne q$, the linear recurrence relation has roots $1$ and $r=q/p$, so $h_i=A+B{r}^i$ for $i\ge 1$. The boundary equation gives $B=r(1-A)$. If $p<1/2$, then $r>1$ and boundedness of probabilities forces $B=0$, hence $A=1$. If $p=1/2$, the repeated-root solution is $A+Bi$; boundedness again gives $B=0$ and the boundary equation gives $A=1$.

For $p>1/2$, $r<1$ and the nonnegative harmonic candidate ${\tilde{h}}_i=r^{i+1}$, together with ${\tilde{h}}_0=1$, satisfies every first-step equation. Hitting-by-time-$n$ probabilities start from zero off the target and iterate those equations, so by induction they are bounded above by this candidate. Taking their increasing limit gives $h_i\le r^{i+1}$. But the general bounded solution has limit $A\ge 0$ as $i\to \infty$, forcing $A=0$ under that bound. ThusIn particular the transient exponent is $i+1$, fixed by the original boundary equation.

The probability of a subsequent return to zero when starting there is $f_{00}=q+p{h}_1$. It is one for $p\le 1/2$, so the Markov chain is recurrent there. For $p>1/2$ it equals $q+p(q/p{)}^2=q/p<1$, so the irreducible Markov chain is transient.

For a stationary probability vector $\pi$, balance at zero and one gives ${\pi }_1=(p/q){\pi }_0$. Subtracting adjacent balance equations givesso this initial ratio propagates and ${\pi }_j={\pi }_0(p/q{)}^j$. It can be normalized only when $p<q$. In that caseDirect substitution verifies all balance equations, including ${\pi }_0=q{\sum }_i{\pi }_i{q}^i$. An irreducible Markov chain admitting this stationary probability law is positive recurrent, with mean return time $1/{\pi }_0=q/(q-p)$. At $p=q$ the linear recurrence relation and boundary relation force every ${\pi }_j$ equal; no such nonzero sequence is summable. The recurrent chain is therefore null recurrent. In summary,