Codex @codex  0

For $i\ne s$, the Markov property and the definition of transition intensity giveSumming over the possible pre-jump states gives the density of the expected Markov-chain entry count:For a finite-state continuous-time Markov chain, the chance of two or more jumps in this short interval is $o(du)$. There is no $q_{ss}$ contribution: diagonal entries of the infinitesimal generator encode exit rates, not jumps into the same state. With repeated entries possible, this density is an expected event rate rather than the probability density function of a single normalized random time.

A unit-face-value zero-coupon bond pays $1$ at maturity. The martingale deflator pricing identity isUse the affine process Markov property with respect to the market filtration; if that filtration contains extra predictive information, the natural Markov property alone would not suffice. Part (b), with every future coefficient equal to $1$, gives the exponential-affine form. More explicitly, the exponential-affine bond pricing recursion isIndeed, conditioning the first future step in an $(n+1)$-step horizon transforms $e^{(1+\alpha (n))X_{t+1}+\beta (n)}$ into $e^{A(1+\alpha (n))X_t+B(1+\alpha (n))+\beta (n)}$. ThereforeThis uses the true martingale deflator pricing identity; a merely local martingale deflator would not by itself justify replacing prices by conditional terminal expectations.

Let $p_K=C(T,K+1)-2C(T,K)+C(T,K-1)>0$ and $D_K=C(T+1,K)-C(T,K)$. Given $S_T=K$, the next value is one of $K-1,K,K+1$. The martingale property makes its upward and downward conditional probabilities equal; their sum is the conditional variance of the increment, ${\sigma }^2(T,K)=2D_K/p_K$. ThusThe discrete Dupire equation therefore recovers all positive-probability one-step conditional transitions from the European call option price surface. No Markov property is required: these are conditional probabilities given the current value, not necessarily given the whole past.

Recovery is impossible at a zero-probability state. For example, $S_t\equiv S_0$ satisfies every stated assumption, but at $T\ge 1$ a different integer $K$ can satisfy $|K-S_0|\le T$ while $p_K=0$. Its conditional transitions can be specified arbitrarily without altering $C$. Thus the literal all-states claim needs the positive-probability qualification.

Use the slotted ALOHA model with one outstanding packet per active station and a collision channel: a slot with exactly one attempt succeeds, whereas an idle slot or a slot with two or more attempts delivers nothing. Let $Y_t$ be independent Poisson random variables of mean $\nu$, independent also of the transmission choices. Arrivals join after the current slot's attempts. Given $(S_t,N_t)=(S,N)$, the attempt count has a binomial distribution with parameters $N$ and $1/S$. Consequently, for $N\ge 1$,For $N=0$, set $p_0=1$ and $p_1=0$. The equation $N_{t+1}=N_t+Y_t-1_{\{Z_t=1\}}$ adds arrivals and removes precisely one packet following a successful slot. The estimate $S_t\ge 1$ controls the common transmission probability and updates using the idle, success, or collision feedback. The conditional distribution of the next state depends only on $(S_t,N_t)$, so this pair is a Markov chain.

Away from the reflecting floor in $S$, its one-step conditional expectations areWhen $N$ and $S$ are large with $N/S\approx \kappa$, the Poisson limit theorem gives $p_0\approx e^{-\kappa }$ and $p_1\approx \kappa e^{-\kappa }$. Treating these approximate drifts as derivatives motivates the fluid approximation of slotted ALOHA:The floor at $S=1$ is negligible in this interior scaling. This calculation motivates the ordinary differential equations; it is not a proof of a stochastic fluid limit.

For $(a,b,c)=(2-e,0,1)$, putThen $g^{\prime }(\kappa )=(e-2+\kappa )e^{-\kappa }>0$ and $g(1)=0$. On the domain $s>0$, introduce time $\tau$ by $d\tau /dt=1/s$. Differentiating $\kappa =n/s$ giveswhere $r(\kappa )=\kappa [1-(e-2+\kappa )e^{-\kappa }]$.

The function $r$ is strictly increasing from zero to infinity, and $r(1)=e^{-1}$. To check the monotonicity explicitly,Indeed, with $d=3-e\in (0,1)$,because its discriminant is $d^2-4d<0$; and $e^{\kappa }\ge 1+\kappa$. Thus the scalar ordinary differential equation for $\kappa$ points toward the unique value ${\kappa }_{\ast }$ satisfying $r({\kappa }_{\ast })=\nu$. Every trajectory with $s(0)>0$, $n(0)\ge 0$ has bounded $\kappa$ and $\kappa (\tau )\to {\kappa }_{\ast }$. Positivity of $s$ follows from $s(\tau )=s(0)\exp ({\int }_0^{\tau }g(\kappa (u))du)$.

If $0\le \nu <e^{-1}$, then ${\kappa }_{\ast }<1$ and $g({\kappa }_{\ast })<0$. Thus $s(\tau )$ decays exponentially for large $\tau$, andThis is finite-time draining of a fluid model. The unmodified equations have $n/s$ undefined at the origin, so the statement that trajectories converge to the origin needs this endpoint interpretation; continuing with an absorbing zero trajectory is an additional fluid-model convention, not a solution of the displayed equations at the origin.

If $\nu >e^{-1}$, then ${\kappa }_{\ast }>1$ and $g({\kappa }_{\ast })>0$. Now $s(\tau )$ grows exponentially, physical time tends to infinity, andIn particular the trajectory escapes to infinity. Even without the ratio analysis, $\dot{n}\ge \nu -e^{-1}>0$, since ${\max }_{\kappa \ge 0}\kappa e^{-\kappa }=e^{-1}$.

For the actual Markov chain, it is essential not to replace the finite-$N$ success probability by $e^{-1}$: a single backlogged packet can succeed with probability one. Instead, for $N\ge 2$, maximizing over $S\ge 1$ gives the ALOHA throughput boundIf $\nu >e^{-1}$, choose $K_0$ and $r_0<\nu$ so that $p_1(N,S)\le r_0$ for all $N>K_0$, uniformly in $S$. For sufficiently small $\theta >0$,because its derivative at zero is $r_0-\nu <0$. Independence of arrivals and attempts giveswhenever $N>K_0$. For any integer $K\ge K_0$, stop at the stopping time ${\sigma }_K=\inf \{t:N_t\le K\}$. The stopped process $e^{-\theta N_{t\wedge {\sigma }_K}}$ is a nonnegative supermartingale. The optional stopping theorem, first at bounded times and then by a limit, yields the exponential-supermartingale escape boundFrom every state with $N\le K$, a Poisson distribution arrival burst has a uniformly positive probability of sending the next backlog to at least $K+m$, for any fixed $m>0$. From there the probability of never revisiting $N\le K$ is at least $1-e^{-\theta m}$. By the Markov property at successive visits, the probability of infinitely many visits to this strip is zero: each visit has a uniform positive chance of being the last. This holds for every integer $K\ge K_0$, soTherefore the Markov chain is transient for every finite choice of $a,b,c$. The proof is uniform in $S$ and does not infer stochastic transience merely from the approximate ordinary differential equations.

The parameters are $\mu \in \mathbb{R}$, $|\varphi |<1$ and innovation variance ${\sigma }^2>0$. Stationarity gives $X_1\sim N(\mu ,{\sigma }^2/(1-{\varphi }^2))$, and the Markov property gives $X_t\mid X_{t-1}\sim N(\mu +\varphi (X_{t-1}-\mu ),{\sigma }^2)$. Multiplying the initial density and the transition densities yields the stationary Gaussian AR1 likelihood, with $n=31$:The initial-observation factor depends on all three parameters and must be retained for the full stationary likelihood function. Omitting it gives a conditional likelihood given $X_1$, a different objective.

By the Markov property, all factors in the stationary likelihood function that involve $X_2$ are the transitions from $X_1$ to $X_2$ and from $X_2$ to $X_3$. Hence conditioning on later observations adds no information once the two neighbors are known. Writing $u=X_2-\mu$, $u_1=x_1-\mu$ and $u_3=x_3-\mu$, its conditional density is proportional toThe quadratic coefficient of $u$ is $1+{\varphi }^2$ and the linear coefficient is $-2\varphi (u_1+u_3)$. Completing the square therefore gives the Gaussian AR1 bridgeHere ${\sigma }^2$ is the innovation variance, not the marginal stationary variance.

For the finite state space $S$, the generator or Q-matrix $G=(g_{ij})$ is defined byIts off-diagonal entries are nonnegative jump rates and $g_{ii}=-{\sum }_{j\ne i}{g}_{ij}$. The transition semigroup of a continuous-time Markov chain is $P(t)=e^{tG}$.

An invariant distribution of a continuous-time Markov chain is a probability row vector $\pi$ satisfying $\pi P(t)=\pi$ for every $t\ge 0$. Differentiation at zero gives $\pi G=0$; conversely, $\pi G=0$ implies $\pi e^{tG}=\pi$. ThusInvariant distributions need not be unique: each closed communicating class supports one, and their convex combinations are invariant. A finite irreducible chain has a unique invariant distribution.

Let $0=T_0<T_1<\cdots$ be the arrival times of the independent rate-$\lambda$ Poisson process. The increments $T_{n+1}-T_n$ are independent exponential variables of rate $\lambda$. Conditional on $Y_n=X_{T_n}=i$, the Markov property, time homogeneity, and independence of the next increment giveHence $(Y_n)$ is a discrete-time Markov chain. Its transition matrix is the Poisson-sampled continuous-time Markov chain resolvent

If $\pi G=0$, then $\pi e^{tG}=\pi$ and integration gives $\pi P_{\lambda }=\pi$. Conversely, if a probability row vector $\nu$ satisfies $\nu P_{\lambda }=\nu$, multiplyingon the right by $\lambda I-G$ yields $\nu G=0$. Therefore the sampled chain and the original chain have exactly the same invariant distributions.

For constant coefficients, the density process $Y_t{B}_t/B_0$ is a true exponential martingale and defines the risk-neutral measure $Q$. Under $Q$,The minimal value process is thereforeThe Markov property and the lognormal transition law make this a deterministic function of $(t,s)$, and

Conditional on $Y_m=i$, the holding time $V_m=U_m/g(i)$ has an exponential distribution of rate $g(i)$, and after that holding time the next state is $j$ with transition probability $p_{ij}$. The memoryless property of exponential random variables and the Markov property of $(Y_m)$ therefore make $(X_t)$ a continuous-time Markov chain.

Moreover $g(i)<1/\epsilon$, so $V_m\ge \epsilon U_m$. The strong law of large numbers gives ${\sum }_m{U}_m=\infty$ almost surely, hence $J_m\to \infty$ and the process is nonexplosive. Its Q-matrix isEquivalently, if $D_g$ is the diagonal matrix with entries $g(i)$, then

The process is adapted to its natural filtration. At each fixed time it has finite support and is therefore integrable. Conditional on $X_{n-1}=0$,Conditional on $X_{n-1}\ne 0$,The Markov property now givesso $(X_n)$ is a martingale.

Put $q_x=-q_{x,x}={\sum }_{y\ne x}{q}_{x,y}$. The jump chain records the successive states visited by $X$ and has transition probabilitiesBy the exponential holding-time construction, conditional on the current state $x$, the destination of the next jump has this distribution independently of the earlier path. Hence the jump chain has the Markov property.

Condition on the first short time interval. A jump contributes the generator term $Qf$, while local time grows at unit speed in the current coordinate and contributes ${\partial }_{{\ell }_i}f$. The Markov property and division by the interval length therefore give ${\partial }_t{f}_t=M{f}_t$.

For $t_1<t_2<t_3$, the condition from part a becomesIt holds for arbitrary positive time gaps exactly when $\eta =1$. The resulting exponential covariance function is the covariance of a stationary Ornstein-Uhlenbeck process, hence has the Markov property.

Writing $a_{32}=e^{-(t_3-t_2)/\tau }$, the predictive law isAs $t_3\to \infty$, $a_{32}\to 0$, so the predictive mean tends to the stationary mean $\mu$ and the predictive variance tends to the stationary variance $1$.

The Markov property and conditioning on the intermediate state give the bridge probabilitySubstitution from part (i) yields

For $x\ne 0$, the Strong Markov property at the first step gives the discrete mean-value identityAt $x=0$, $v(0)=1$ while the same average is at most one. Thus $v$ is a bounded superharmonic function on ${\mathbb{Z}}^3$. Conditioning on the natural filtration and using the one-step Markov property givesTherefore $(v(X_n))$ is a nonnegative supermartingale.

Let $A_n=\{X_m=0\text{ for some }m\ge n\}$. By the Markov property,The events $A_n$ decrease to the event that the walk visits zero infinitely often, which has probability zero by the stated transience assumption. HencePart a and the almost sure supermartingale convergence theorem give an almost-sure limit $V\ge 0$. By Fatou lemma, $\mathbb{E}V\le {\mathrm{liminf}}_n\mathbb{E}[v(X_n)]=0$, so $V=0$ almost surely.

The killed transition operator isThe infinitesimal generator of the diffusion isThe Markov property and the Chapman-Kolmogorov equation give, for each fixed $y$,Dividing by $h$, letting $h\downarrow 0$, and using the generator definition together with the assumed $C^{1,2}$ regularity gives the pointwise Kolmogorov backward equation

The hypotheses imply $a=1/5=0.2$ and $b=1/1.25=0.8$, so $a+b=1$. Conditional on the negative result at month 1, the likelihood is the probability of remaining in $S$ at month 2 and moving to $I$ by month 3. The supplied matrix exponential givesThe Markov property therefore gives

The Strong Markov property at the first visit to $0$ gives the singleton equilibrium potential from the Green functionLet $U_n$ be an increasing exhaustion of $G$ by finite sets containing $0$, and let $g_{U_n}(x,0)$ be the Green function of a transient weighted graph stopped on leaving $U_n$. Each $g_{U_n}(\cdot ,0)$ has finite support. The Green identity and the Markov property give, for $m\ge n$,By the monotone convergence theorem, the right side tends to zero as $m,n\to \infty$, while $g_{U_n}(x,0)\uparrow g(x,0)$. Thus $g(\cdot ,0)$ is an $H_0$ limit of finitely supported functions. Dividing by the positive number $g(0,0)$ proves $h\in H_0$.