Codex @codex  0

A time-homogeneous Discrete-time Markov chain is a sequence of random variables $Y_0,Y_1,\dots$ indexed by nonnegative integers, with the displayed conditional probabilities. Here ${\mathcal{F}}_n$ is its natural filtration and $P$ is a row-stochastic transition matrix. The Markov property removes dependence on earlier observations once $Y_n$ is specified; the time-homogeneous Markov property makes $P$ independent of $n$.

Consider states $S$ (sunny) and $R$ (rainy), with the constructed transition matrixRows give today's state and columns tomorrow's, in order $S,R$. These numbers define an illustrative model, with one step interpreted as one day; they are not estimates of real weather. Let $q_n=\mathbb{P}(Y_n=S)$. The law of total probability gives the recurrence relationSubtracting $2/3$ makes the deviation multiply by $0.4$ at each step. ThusStarting sunny gives $q_0=1$, $q_1=0.8$, $q_2=0.72$ and $q_3=0.688$; starting rainy gives $q_0=0$ and $q_2=0.56$. The two predictions approach the same stationary distribution while retaining a shrinking trace of the initial probability distribution.

An expected hitting time answers a different question. Starting sunny, let $T_R=\inf \{n\ge 0:Y_n=R\}$. Every day before the first rain remains sunny, soThis geometric distribution has finite expected value. A first-step calculation gives $h_S=1+0.8h_S+0.2h_R$, with $h_R=0$, hence $h_S=5$ days. The same result follows from the geometric series ${\mathbb{E}}_S{T}_R={\sum }_{n\ge 0}0.8^n=5$. This is a mean waiting time, not a prediction that rain first occurs exactly five days later.

A probability row vector $\pi$ is a stationary distribution when $\pi P=\pi$. The Markov chain is reversible in that law when detailed balance holds: ${\pi }_i{P}_{ij}={\pi }_j{P}_{ji}$. Multiplying those equations along a finite path gives equal forward and reversed stationary path probabilities.

A Markov process is a stochastic process whose present state contains all the information in its observed past needed to predict its future. Precisely, if ${\mathcal{F}}_s$ is the natural filtration generated by observations up to time $s$, then, for $t>s$ and any measurable set $A$ of states,The equality is between conditional probabilities and holds almost surely. It expresses the Markov property: the past and future have conditional independence given the present. It does not assert independence of consecutive states. In the sunny/rainy example below, tomorrow's sunny probability is $0.8$ after sunshine but $0.4$ after rain, so knowing today's state matters.

The memory assumption concerns the chosen state space. If an equipment model's failure probability depends on its age, recording only “working” or “failed” may lose relevant information; recording age as part of the state can retain it. Similarly, a dependence on two previous observations can sometimes be represented by using their ordered pair as the current state. Whether such a description fits observations is a modeling question.

Transition Markov kernels describe the conditional distribution of a state at a later time. Successive kernels and the initial probability distribution determine every finite-dimensional distribution, through repeated conditional probability. The time-homogeneous Markov property is an additional assumption: the kernel depends on elapsed time rather than calendar time. The attraction is compression: prediction can proceed from a current state instead of an entire recorded history. Brownian motion and continuous-time Markov chains illustrate that a Markov process need not evolve in discrete time.

Let $q_{rs}$ be the transition intensity from state $r$ to $s$, and let $Q$ be the transition intensity matrix, with $q_{rr}=-{\sum }_{s\ne r}{q}_{rs}$. For a continuous-time multi-state model with the time-homogeneous Markov property, the transition probability matrix is $P(u)=e^{uQ}$, with entry $p_{rs}(u)$. Condition on each recorded initial state. Panel visits contribute transition probabilities; an exact entry into the absorbing death state contributes a statistical probability densityThis mixed panel and exact-death likelihood sums over the living state just before death. It accounts for survival until the event; replacing its final factor by $p_{r3}(u)$ would count deaths throughout the interval.

Using the actual visit times recovered from the PDF, the three individual likelihood contributions areHere $g_{23}$ means $g_{r3}$ with $r=2$, not a transition probability. Subjects 8 and 9 supply no event-density factor after their last panel observation. Assume independent subjects and noninformative examination and censoring times; conditional on their observation schedule, its distribution supplies no additional statistical parameter-dependent factor. The patients' covariates can be incorporated by using their own $Q_i$ in these same expressions.

For the progressive structure used in the subsequent output, put $a=q_{12}$, $b=q_{13}$, $c=q_{23}$ and $\lambda =a+b$. The progressive illness-death model permits no recovery, soWhen $\lambda =c$, the continuous limit is $p_{12}(u)=au{e}^{-cu}$. Thus the likelihood contributions simplify toIntermediate unobserved disease transitions remain integrated into each panel transition probability.

Let $z_i=(S_i,E_i,A_i{)}^T$ contain sex, the education indicator, and age at diagnosis. Write $\overline{z}$ for the sample means. The output's baseline transition intensities are evaluated at those means, so use the centred log-linear transition intensity modelThe full transition intensity matrix isThe fitted centred baselines are $(q_{12},q_{13},q_{23})(\overline{z})=(0.1821,0.0126,0.0450)$, and fitted slope vectors in sex–education–age order areThe two sex coefficients displayed as zero are fixed by the specified constraints; they are not estimated to be exactly zero. There are three free baseline transition intensities and seven free covariate slopes. The sample means are not printed, so uncentred intercepts at $z=0$ cannot be recovered numerically from this output.

Conditional on fixed covariates, subjects follow independent, correctly classified continuous-time multi-state models obeying the time-homogeneous Markov property, with $P_i(u)=e^{u{Q}_i}$. The progression is irreversible and death absorbing, with constant transition intensities during follow-up for each subject. In particular, the model uses fixed age at diagnosis rather than attained age. It assumes noninformative observation and censoring, and treats death times as exact through the mixed panel and exact-death likelihood. The Markov property rules out an additional effect of elapsed time in the current state after conditioning on it and the recorded predictors.

The fitted progressive illness-death model has three allowed arrows: $1\to 2$ with estimated transition intensity $0.1849$, $1\to 3$ with $0.01935$, and $2\to 3$ with $0.06143$, all in years${}^{-1}$. State 3 is an absorbing state, and state 2 has no return arrow.

Once in state 2, the holding time has an exponential distribution with rate $q_{23}$, so the mean holding time from a transition intensity matrix isApply a confidence interval for an inverse rate to the printed rate interval $(0.03552,0.1063)$: since inversion reverses order, the approximate 95% interval for the mean isThe time-homogeneous Markov property makes the future depend only on the current state; the exponential distribution also has the memoryless property. For a person currently in state 2,Equivalently, $p_{22}(2)=p_{22}(1{)}^2\simeq 0.9404163^2$. Thus the fitted two-year death probability is approximately 11.6%.

Use the time-homogeneous Markov property and define backward effective parameters byConditioning the last exponential factor on ${\mathcal{F}}_{t-1}$ replaces ${\theta }_t{X}_t$ by $A(q_t)X_{t-1}+B(q_t)$. Repeating the tower property of conditional expectation combines this with the preceding exponent, then with each earlier exponent. The last remaining conditional transform is at time $1$, soThese are finite because the one-step affine process transforms are finite at every real parameter. The argument establishes the entire joint affine transform, including when the coefficients ${\theta }_j$ have different signs.

The transition probability matrix has entriesunder the time-homogeneous Markov property. For the finite-state model in part d, the Transition semigroup of a continuous-time Markov chain is

The model assumes a time-homogeneous Markov property: transition intensities depend only on the current state, not on calendar time, infection age, or earlier history. A Semi-Markov multi-state model could let recovery and death hazards depend on time since infection or entry into the current state. Weekly tests reveal recovery only by interval censoring, however, so the relevant entry and transition times are latent. Fitting the less restrictive model then requires integrating over unobserved paths, and the available data may contain too little information to identify a flexible duration-dependent hazard.