Codex @codex  0

Communication is reflexive because $(P^0{)}_{ii}=1$, and symmetric by its definition. If $i\leftrightarrow j$ and $j\leftrightarrow k$, choose positive-probability paths $i\to j$ in $m$ steps and $j\to k$ in $n$ steps. The Chapman-Kolmogorov equation givesCombining the reverse paths gives a positive-probability path $k\to i$ as well. Thus communication is transitive and is an equivalence relation.

To transfer recurrence, first establish the recurrence criterion by return probabilities. Put $r_i={\mathbb{P}}_i(T_i^{+}<\infty )$ and let $N_i={\sum }_{n\ge 0}{1}_{\{X_n=i\}}$ under ${\mathbb{P}}_i$. After each return the Strong Markov property restarts the chain at $i$, so ${\mathbb{P}}_i(N_i\ge k+1)=r_i^k$. ThereforeFor $r_i<1$ this equals $(1-r_i{)}^{-1}$; for $r_i=1$ it is infinite. The expectation identity follows by summing the nonnegative indicators, so remains valid when the expectation is infinite. Consequently $i$ is a recurrent state exactly when its return-probability series diverges.

If $i\leftrightarrow j$, choose $a,b$ with $(P^a{)}_{ji}>0$ and $(P^b{)}_{ij}>0$. For every $n\ge 0$,Summing over $n$, recurrence of $i$ forces the return-probability series for $j$ to diverge. The proved criterion shows $j$ is recurrent, so recurrence is a property of the communicating class.

The Markov property and the Chapman-Kolmogorov equation giveFor each intermediate state, one first uses the next-step transition probability and then the following one; the sampled history adds no information once $X_{2n}$ is known. Thus $W$ is a homogeneous Markov chain with transition matrix $P^2$ and initial law $\lambda$. In particular,If zero is not a state of $S$, this probability is zero.

On a compact metric space $X$, a conservative Feller semigroup is a strongly continuous semigroup of positive linear maps on $C(X)$ satisfying $P_{t}1=1$. Positivity means $f\ge 0⟹P_{t}f\ge 0$. These conditions imply sup-norm contraction. Conversely, for a contraction semigroup with $1\in D(L)$ and $L1=0$, differentiation on the generator domain gives $\frac{d}{dt}{P}_{t}1=P_{t}L1=0$, so $P_{t}1=1$. On the real space $C(X)$, if $0\le f\le 1$, then $\Vert 1-P_{t}f{\Vert }_{\infty }\le \Vert 1-f{\Vert }_{\infty }\le 1$, implying $P_{t}f\ge 0$. Scaling proves positivity for every nonnegative $f$. This is the unital contraction positivity criterion. For complex $C(X)$ the same conclusion follows because each functional $f\mapsto P_{t}f(x)$ has norm one and value one on $1$, hence is a positive linear functional.

For each $t,x$, the Riesz-Markov-Kakutani representation theorem represents that functional by a unique Borel probability measure $p_t(x,dy)$:Continuity of $P_{t}f$ gives weak continuity of $x\mapsto p_t(x,\cdot )$. Approximating indicators of open sets increasingly by continuous functions proves Borel measurability in $x$; a monotone-class argument then gives a Markov kernel. The semigroup property and uniqueness of representing measures yield $p_0(x,\cdot )={\delta }_x$ and the Chapman-Kolmogorov equationThese are the required transition probabilities.

An invariant probability measure for a semigroup satisfies ${\int }_X{P}_{t}fd\mu ={\int }_{X}fd\mu$ for every $t\ge 0$ and $f\in C(X)$. Equivalently its distribution is preserved by the Markov kernel. For $1\le p<\infty$, Jensen inequality gives $|P_{t}f{|}^p\le P_t(|f{|}^p)$ pointwise. Integrating and using invariance provesAlso $\Vert P_{t}f-f{\Vert }_{L^p(\mu )}\le \Vert P_{t}f-f{\Vert }_{\infty }\to 0$. Continuous functions are dense in these $L^p$ spaces for a finite Borel measure on a compact metric space, so the contraction extends to a strongly continuous semigroup on $L^p(\mu )$.

For real $f$ with $f,f^2\in D(L)$, the kernel form of Jensen inequality gives $P_t(f^2)-(P_{t}f{)}^2\ge 0$, with equality at $t=0$. Take its right derivative to obtain the generator square inequalityThis is nonnegativity of the associated carré du champ.

Each time-average functional ${\varphi }_n$ is positive with ${\varphi }_n(1)=1$, hence represents a Borel probability measure ${\mu }_n$. The compactness of probability measures on a compact metric space gives a subsequence converging weakly to a probability measure $\mu$. Explicitly, $C(X)$ is a separable Banach space, so Banach-Alaoglu theorem and a countable dense family give a subsequence converging on every continuous function, rather than merely a net. For fixed $t\ge 0$, the semigroup property gives the Krylov-Bogolyubov time-average argument:Its absolute value is at most $2t\Vert f{\Vert }_{\infty }/n$. Pass to the subsequential limit, observing that $P_{t}f\in C(X)$, to get ${\int }_X{P}_{t}fd\mu ={\int }_{X}fd\mu$. Thus the limiting measure is invariant.

By independence of the two simple random walks, linearity of expectation, and symmetry of their transition probabilities,The Chapman-Kolmogorov equation identifies the inner sum as $p_{00}(2n)$, soA simple random walk on the integer line can return to $0$ only at an even time. Since it is recurrent at $0$, part (a) givesTherefore $\boxed{\mathbb{E}Z=\infty }$.

The killed transition operator isThe infinitesimal generator of the diffusion isThe Markov property and the Chapman-Kolmogorov equation give, for each fixed $y$,Dividing by $h$, letting $h\downarrow 0$, and using the generator definition together with the assumed $C^{1,2}$ regularity gives the pointwise Kolmogorov backward equation

A row-stochastic matrix has nonnegative entries and each row sums to one. For a time-homogeneous Discrete-time Markov chain, its transition matrix has entriesIf a state has zero probability at time $n$, this row is specified by the model's Markov kernel, rather than recovered by conditioning on that zero-probability event. Rows index departure states and columns arrival states. Writing the current probability distribution as a row vector ${\mu }_n$ givesThe row-sum condition ensures that matrix multiplication preserves total probability. Under a column convention the evolution uses the transpose; the convention must accompany the formula.

For the sunny/rainy transition matrix, direct matrix multiplication givesIn particular, $(P^2{)}_{SS}=0.8\times 0.8+0.2\times 0.4=0.72$. The two terms sum the disjoint paths $S\to S\to S$ and $S\to R\to S$. Requiring both subsequent days to be sunny has probability $0.64$; requiring only the second day to be sunny permits the extra path through rain.

Generally $(P^n{)}_{ij}$ is the $n$-step transition probability. The identity $P^{m+n}=P^m{P}^n$ is the discrete-time Chapman-Kolmogorov equation: sum over the intermediate state. This makes matrix powers useful for forecasting many steps without enumerating every path individually.

For a time-homogeneous continuous-time Markov chain, $P_{ij}(t)={\mathbb{P}}_i(X_t=j)$ and the matrices satisfy $P(s+t)=P(s)P(t)$, the Chapman-Kolmogorov equation. On a finite state space with Q-matrix $Q$, the matrix exponential gives $P(t)=e^{tQ}$.

The distinction from a discrete-time transition matrix is the elapsed-time parameter: $P(t)$ describes where the process is after $t$ time units, allowing any number of intervening jumps. Each $P(t)$ is a stochastic matrix, with $P(0)=I$. A Q-matrix instead records instantaneous rates and has row sums zero. Sampling at a fixed spacing $\Delta$ yields a Discrete-time Markov chain with transition matrix $P(\Delta )$. See Hao Wu's MIT lecture 21, transition semigroups and lecture 22, infinitesimal rates.