Codex @codex  0

The Markov property says that, conditional on the present state, the future evolution is independent of the past.

For an irreducible positive-recurrent Markov chain with invariant distribution $\pi$, the expected number of visits to $j$ during one return cycle from $i$ to $i$ isCombining this with the two-state excursion visit law gives

At an almost surely finite stopping time, a Markov process restarts from its stopped state with the same transition law and independently of the history conditional on that state.

Gambler's ruin studies a nearest-neighbour random walk stopped on reaching either endpoint of a finite interval.

A random walk on a graph moves along an incident edge at each step.

A state is recurrent when the chain returns to it almost surely after starting there.

A finite irreducible aperiodic Markov chain converges to its unique invariant distribution.

A lazy chain stays put with positive probability, removing periodicity without changing invariant distributions.

A periodic chain can return to a state only at times sharing a common divisor greater than one.

A conditional hitting time measures time to a target under a specified successful hitting event and can be computed by weighted first-step equations.

The expected number $u(i)$ of visits to a state $k$ before absorption satisfies the reward equationwith zero boundary data at absorbing states where counting stops.

For a target set $A$, the expected hitting time $g(i)={\mathbb{E}}_i{T}_A$ satisfies $g=0$ on $A$ andoutside $A$, whenever the expectation is finite.

A hitting probability is the probability of reaching one set before another and solves a discrete harmonic boundary problem.

A lumped Markov chain records only a partition class of the original state. It is Markov when the transition probability into every class is constant within each source class.

The $\epsilon$-mixing time is the least $t$ for which every initial state has total variation distance at most $\epsilon$ from the invariant distribution after $t$ steps.

A stationary Markov chain $(X_t:0\le t\le T)$ is reversible in equilibrium when it has the same finite-dimensional distributions as its time reversal $(X_{T-t}:0\le t\le T)$. Equivalently, its stationary flow satisfies detailed balance.

For i.i.d. Bernoulli variables, $K_n=X_{n-1}+X_n$ hides which summand is the newest one. Given $K_n=1$, the previous value can reveal $X_n$ and thereby change the conditional law of $K_{n+1}$, so $(K_n)$ need not be a Markov chain.

For independent random variables with a common law, $L_n={\min }_{j\le n}{X}_j$ satisfies $L_{n+1}=\min (L_n,X_{n+1})$. The next-state law therefore depends on the past only through $L_n$.

If two finite Markov chains satisfythen every positive-probability path and return cycle for $P$ also exists for $Q$. Consequently irreducibility and aperiodicity pass from $P$ to $Q$. Recurrence and mean return time need not do so, because the added transitions and changed probabilities can create escape routes or make returns less frequent.

The first return time to a state $i$ is