Codex @codex  0

An affine process is a time-homogeneous Markov process whose conditional moment-generating function is exponential-affine in its starting state. In discrete time one writes $\log {\mathbb{E}}_x{e}^{\theta X_1}=A(\theta )x+B(\theta )$ on the finite exponential moment domain. Continuous-time definitions allow the functions to depend also on elapsed time.

For a Markov process generator and suitable real functions, this bilinear expression records the local quadratic energy. Positivity of $\Gamma (f,f)$ follows by differentiating $P_t(f^2)\ge (P_{t}f{)}^2$. For $L=D^2-xD$ it is $f^{\prime }{g}^{\prime }$, giving the Gaussian Dirichlet energy after integration against the invariant measure.

A controlled Markov process has transition probabilities or rates depending on its current state and the chosen control action. A Markov policy selects an action from the current state and possibly the time. Once such a policy is fixed, the induced state evolution is a Markov process. For general history-dependent controls, the state alone need not remain Markov. A dynamic programming operator optimizes over the available actions at each state.

For a maturity-differentiable Heath-Jarrow-Morton model forward field with the integrability needed for stochastic differentiation, the short rate $r_t=f(t,t)$ has the displayed dynamics. Indeed changing observation time uses the forward-rate stochastic differential equation, while changing maturity adds ${\partial }_{T}f(t,t)dt$. Under a risk-neutral measure, $\alpha (t,T)=\sigma (t,T){\int }_t^T\sigma (t,u)du$, so $\alpha (t,t)=0$. The remaining drift depends on the diagonal slope of the whole forward curve, which need not be determined by $r_t$; a Markov process for the short rate is an additional modelling restriction. Without sufficient maturity regularity, the diagonal need not admit this classical Itô process formula.

For a killed Markov process and a positive harmonic function $h$, its Doob h-transform weights the path law through a stopping time $\tau$ by $h(X_{\tau })/h(X_0)$, with localization when necessary. It describes conditioning on a boundary behaviour of probability zero. In the complex upper half-plane, $h(z)=\mathrm{Im}z$ conditions killed planar Brownian motion to remain in that half-plane.

For a Markov process with infinitesimal generator $L$ and a suitable test function $f$, the displayed process is a local martingale. With integrability it is a true martingale and yields the corresponding expectation identity at admissible stopping times. For bounded $f,Lf$ in the semigroup generator domain, $P_{t}f-f={\int }_0^t{P}_{s}Lfds$. The Markov property applies this identity conditionally over each interval, proving the martingale increments have zero conditional expectation. Localization gives the local version. For a continuous diffusion, these identities are the martingale problem for $L$.

Particles on a countable lattice attempt jumps at the rates $p(x,y)$, and an attempt into an occupied site is suppressed. Thus each site contains at most one particle and the number of particles is conserved when finite. Here ${\eta }^{xy}$ interchanges the occupations at $x$ and $y$. Bounded nearest-neighbour rates give a well-defined Markov process. A symmetric jump kernel gives the symmetric simple exclusion process; an unequal left-right kernel gives the asymmetric simple exclusion process.

Consider a finite-duration Markov process with discrete, time-even states coupled to one thermal reservoir at positive temperature $T$. Let $P_F[\gamma ]$ include the initial probability ${\rho }_0(x_0)$ of the sample trajectory $\gamma$. Define $P_R$ by reversing the control protocol and initializing it with the actual forward final probability distribution ${\rho }_{\tau }$. Write $\Theta \gamma$ for the same states visited in reverse time order. Require normalized path laws with matching probability support: a path has positive forward weight exactly when its reverse has positive reverse weight. This excludes irreversible forbidden transitions and missing initial states.

Under local detailed balance, with positive heat $Q_{\mathrm{bath}}$ delivered to the thermal reservoir, the full path-weight logarithm isThe endpoint term is the change in stochastic system entropy; the conditional path ratio alone accounts for the thermal reservoir. Here $k_B$ is the Boltzmann constant. Both terms matter even for a short observation. ^[ref]

Define the conjugate reverse functional by ${\Sigma }_R[\eta ]=\ln (P_R[\eta ]/P_F[\Theta \eta ])$, so ${\Sigma }_R[\Theta \gamma ]=-{\Sigma }_F[\gamma ]$. Summing the path identity over paths with a given value gives the exact finite fluctuation theoremfor nonzero denominator, using probability masses or corresponding probability density functions. This ${\Sigma }_R$ is defined by the paired laws: it need not be the total entropy production obtained by independently restarting the thermodynamic construction for the reverse experiment. A generic driven experiment therefore does not imply $p_F(s)/p_F(-s)=e^s$.

A hidden Markov model is a Markov process whose state is latent and observed only through random emissions whose conditional distribution depends on the current hidden state.

For the standard fixed routing loss network with independent Poisson processes, the stationary occupancy law depends on independent holding-time distributions only through their means. The occupancy process alone need not be a Markov process when those holding times lack the memoryless property.

A probability measure is invariant for a Markov process with transition semigroup $P_t$ when starting with that law gives the same law at every later time. Equivalently, $\int P_{t}fd\nu =\int fd\nu$ for every bounded measurable function. Invariant laws may be constructed by weak convergence of probability measures of evolved laws when the Feller semigroup property permits passing the semigroup through the limit, as for the extremal invariant measures of the contact process.

The term Markov chain commonly denotes a Markov process on a finite or countable state space. Time may be discrete, as for a daily record, or continuous, as for a continuous-time Markov chain. Some authors also use “Markov chain” for discrete-time processes on more general state spaces; specifying both time and state space avoids this ambiguity.

For a Discrete-time Markov chain, its initial probability distribution ${\mu }_0$ and transition matrix $P$ specify the probability of each finite path:This is the Markov factorization, obtained by applying conditional probability repeatedly. See Hao Wu's MIT lecture 1, definition and distribution evolution.

For the constructed sunny/rainy Markov chain below, starting sunny makes the path sunny, rainy, sunny have probability $1\times 0.2\times 0.4=0.08$. Starting instead with stationary distribution $(2/3,1/3)$ makes that path's probability $(2/3)\times 0.2\times 0.4=4/75$. The transition probabilities are identical in both experiments; the initial probability distribution distinguishes them.

A Markov diffusion is a continuous-path Markov process whose local evolution is described by a drift vector and diffusion matrix. For sufficiently regular coefficients, its infinitesimal generator is a second-order differential operator and its conditional expectations satisfy a Kolmogorov backward equation.

A Markov process is a stochastic process whose present state contains all the information in its observed past needed to predict its future. Precisely, if ${\mathcal{F}}_s$ is the natural filtration generated by observations up to time $s$, then, for $t>s$ and any measurable set $A$ of states,The equality is between conditional probabilities and holds almost surely. It expresses the Markov property: the past and future have conditional independence given the present. It does not assert independence of consecutive states. In the sunny/rainy example below, tomorrow's sunny probability is $0.8$ after sunshine but $0.4$ after rain, so knowing today's state matters.

The memory assumption concerns the chosen state space. If an equipment model's failure probability depends on its age, recording only “working” or “failed” may lose relevant information; recording age as part of the state can retain it. Similarly, a dependence on two previous observations can sometimes be represented by using their ordered pair as the current state. Whether such a description fits observations is a modeling question.

Transition Markov kernels describe the conditional distribution of a state at a later time. Successive kernels and the initial probability distribution determine every finite-dimensional distribution, through repeated conditional probability. The time-homogeneous Markov property is an additional assumption: the kernel depends on elapsed time rather than calendar time. The attraction is compression: prediction can proceed from a current state instead of an entire recorded history. Brownian motion and continuous-time Markov chains illustrate that a Markov process need not evolve in discrete time.

The transition operators of a time-homogeneous Markov process satisfy $P_{t+s}=P_t{P}_s$ by the Markov property. They preserve constants and positivity. When the generator and function domain permit differentiation, ${\partial }_t{P}_{t}g=L(P_{t}g)$ is the Kolmogorov backward equation. Transition semigroup of a continuous-time Markov chain is the discrete-state special case.

For time-to-maturity coordinates $g_t(x)=f(t,t+x)$ and time-homogeneous deterministic Heath-Jarrow-Morton model volatility, $A(x)=⟨\sigma (x),{\int }_0^x\sigma (u)du⟩$. The displayed evolution contains the maturity shift. Its mild solution is $g_t(x)=g_0(x+t)+{\int }_0^{t}A(x+s)ds+{\int }_0^t⟨\sigma (x+t-s),d{W}_s⟩$. The whole curve is a time-homogeneous Markov process under suitable function-space well-posedness conditions; the current short rate alone need not be a sufficient state.

For a Markov process with reward $f$, the value starting at time $t$ in state $s$ is the supremum of expected rewards over admissible stopping times. In finite discrete time it obeys $V(T,s)=f(s)$ and $V(t,s)=\max \{f(s),PV(t+1,\cdot )(s)\}$ for the transition operator $P$, wherever expectations are well-defined. Statewise absolute integrability of every remaining-horizon reward makes these values finite.

An Ornstein-Uhlenbeck process combines mean reversion with continuing random fluctuations. In the scalar model, its stochastic differential equation iswhere $W$ is a standard Brownian motion. The drift points towards $\mu$: it is negative above $\mu$ and positive below it. The coefficient $\theta$ sets the speed of mean reversion, while $\sigma$ sets the strength of the random forcing. We consider a fixed or normal initial value independent of future Brownian increments, giving a Gaussian process and a Markov process.

For a stipulated example, use $\theta =1$, $\mu =2$, $\sigma =\sqrt{2}$ and the fixed initial value $X_0=6$, in arbitrary consistent state and time units. Then the expected value relaxes as $2+4e^{-t}$ and the variance grows as $1-e^{-2t}$. At time one they are about $3.4715$ and $0.8647$; the limiting variance is one. The sample paths keep fluctuating even after the expected value is close to two. The Ornstein-Uhlenbeck transition distribution describes this evolving uncertainty; Ornstein-Uhlenbeck stationary initialization instead starts with that limiting normal distribution already in place.

A physical motivation is the velocity of a particle undergoing Brownian motion: linear drag removes a persistent initial velocity, while random forcing continually renews it. The original calculation gives a normal distribution for the velocity after subtracting its exponentially decaying initial contribution, with decay rate equal to the friction coefficient divided by the particle's mass.^[ref] A position coordinate in an overdamped harmonic optical trap gives another interpretation of the same scalar equation, with $\theta =k/\zeta$ for spring constant $k$ and drag coefficient $\zeta$. Its physical assumptions and connection to measurements are developed in thermal calibration of an optical trap.

Suppose the forward field is differentiable in maturity with the stochastic integrability needed for differentiation. As time increases, both arguments of $r_t=f(t,t)$ increase. The diagonal short-rate dynamics in the Heath-Jarrow-Morton model are thereforeUnder the HJM drift restriction from part (a), ${\alpha }^Q(t,t)=0$, givingMore explicitly the slope isIt depends on the forward curve and potentially the whole history, not necessarily on $r_t$ alone. Without the stated maturity regularity, the original forward equations do not by themselves guarantee this classical Itô process formula for the diagonal.

The short rate need not be Markov, even with one Brownian factor and deterministic volatility. An explicit example of One Brownian factor does not imply a Markov short rate takes $f_0(T)=0$ and $\sigma (t,T)=T-t$. Its no-arbitrage forward drift is ${\alpha }^Q(t,T)=(T-t{)}^3/2$, soThe continuous rate history determines $W_t$ from the left derivative of $I_t$. Hence for $h>0$,But $W_t$ is not determined by $I_t$: joint normality, $\mathrm{Var}{I}_t=t^3/3$, and $\mathrm{Cov}(W_t,I_t)=t^2/2$ giveThus the conditional mean of the future given the whole rate history cannot be a function of the current rate alone, contradicting the Markov property. Enlarging the state to $(I_t,W_t)$ gives a two-dimensional Markov process; special HJM specifications can instead close on a single short rate.

A Gaussian one-factor short-rate model is most naturally specified under a chosen risk-neutral measure $Q$, not by using a physical drift unaltered in a pricing formula. The Vasicek model hasIts explicit solution is $r_s=\bar{r}+(r_t-\bar{r})e^{-\kappa (s-t)}+\eta {\int }_t^s{e}^{-\kappa (s-u)}d{W}_u^Q$. It is a Markov process, and its conditional mean and variance are $\bar{r}+(r_t-\bar{r})e^{-\kappa (s-t)}$ and ${\eta }^2(1-e^{-2\kappa (s-t)})/(2\kappa )$. Hence all finite-dimensional distributions are Gaussian. The model permits negative rates; Gaussianity does not enforce their positivity.

The unit zero-coupon bond price is $P(t,T)={\mathbb{E}}_Q[e^{-{\int }_t^T{r}_{s}ds}\mid {\mathcal{F}}_t]$. Write $\tau =T-t$ and $B(\tau )=(1-e^{-\kappa \tau })/\kappa$. Stochastic integration of the explicit solution givesThis Gaussian variable has variance ${\eta }^2{\int }_0^{\tau }B(v{)}^{2}dv$. Evaluating its exponential moment gives exponential-affine bond pricingFor example, ${\int }_0^{\tau }B(v{)}^{2}dv={\kappa }^{-2}[\tau -2B(\tau )+(1-e^{-2\kappa \tau })/(2\kappa )]$, which gives the displayed expression by direct simplification. The term structure of interest rates is therefore analytically tractable, and bond-price changes are driven by the current short rate alone.

The Hull-White model replaces the drift by $\alpha (t)-\kappa r_t$ with deterministic $\alpha$. The same calculation givesThis retains the one-factor Gaussian and Markov structure while allowing an exact initial-curve fit through $\alpha$. If $f_0(t)$ is the initial instantaneous forward rate and $r_0=f_0(0)$, differentiating the bond formula shows thatThus calibration flexibility is separated from stochastic mean reversion. Time-dependent calibration generally destroys time stationarity even though the state remains Markov. In the time-homogeneous Vasicek case, a stationary initialization is $r_0\sim N(\bar{r},{\eta }^2/(2\kappa ))$ independent of future increments; its covariance is ${\eta }^2{e}^{-\kappa |t-s|}/(2\kappa )$. A fixed initial short rate does not give a stationary process.