= Solution
A <martingale> describes an observable after its predictable drift has been removed. For a <Markov chain> the drift is the change produced by its transition operator; for <Brownian motion> it is one half of the <Laplacian>. Requiring the resulting processes to be <martingales> for a sufficiently rich class of observables determines the transition law itself. The same identities, combined with <optional stopping>, determine hitting probabilities, expected exit times and recurrence or transience. The duplicated Q5 paragraph in the PDF is a repeated printing of this one essay request.
For a finite or countable-state <Markov chain> with <transition matrix> $P$, define $(Pf)(x)=\sum_yP(x,y)f(y)$ for a bounded test function $f$. The <Markov property> says $\mathbb E[f(X_{n+1})\mid\mathcal F_n]=(Pf)(X_n)$. Consequently
$$
M_n^f=f(X_n)-f(X_0)-\sum_{j=0}^{n-1}\bigl((Pf)(X_j)-f(X_j)\bigr)
$$
is a <martingale>, since each increment has conditional <expected value> zero. Conversely, if these processes are <martingales> for every bounded $f$, taking the conditional expectation of an increment recovers that identity. With $f=\mathbf1_{\{y\}}$ it gives $\mathbb P(X_{n+1}=y\mid\mathcal F_n)=P(X_n,y)$ for every state $y$. Thus, given the initial law, \b[these identities characterize the <Markov chain>], not merely a few of its moments. This is the <discrete-time Markov chain martingale characterization>.
The drift operator is $P-I$. If $(P-I)h=0$ away from a target set, $h(X_{n\wedge T})$ is a <stopped martingale>, where $T$ is the first hit of that set. Under boundedness or suitable <uniform integrability>, the <optional stopping theorem> expresses $h$ in terms of boundary hitting probabilities. Similarly, if $(P-I)v=-1$ away from the target and $v=0$ on it, then $v(X_{n\wedge T})+(n\wedge T)$ is a <martingale>, giving $v(x)=\mathbb E_xT$ when the terminal limit is justified.
For a concrete calculation, take a <simple symmetric random walk> on $\{0,1,\ldots,N\}$ stopped at $T$, its first hit of $0$ or $N$. In the interior, $h(i)=i$ has zero drift, while $v(i)=i(N-i)$ satisfies
$$
\frac12v(i-1)+\frac12v(i+1)-v(i)=-1.
$$
The stopped identity gives $\mathbb E_i(T\wedge n)=v(i)-\mathbb E_iv(X_{T\wedge n})\le v(i)$, proving $T$ finite with finite mean. Bounded convergence in both stopped identities now yields the \b[<gambler's ruin> answers]
$$
\boxed{\mathbb P_i(X_T=N)=i/N,\qquad\mathbb E_iT=i(N-i).}
$$
This illustrates why the integrability justification is part of using a <martingale>, rather than an automatic consequence of writing an equation.
For a finite-state <continuous-time Markov chain> with generator matrix $Q=(q_{ij})$, the corresponding <martingale problem for a continuous-time Markov chain> uses
$$
M_t^f=f(X_t)-f(X_0)-\int_0^t(Qf)(X_s)\,ds,\qquad (Qf)(i)=\sum_{j\ne i}q_{ij}(f(j)-f(i)).
$$
These are <martingales>. Conversely, suppose an adapted <càdlàg> finite-state process satisfies these identities. Let $P_t=e^{tQ}$, fix $T$, and set $g(s,i)=(P_{T-s}f)(i)$. Then $\partial_sg+Qg=0$. Apply deterministic integration by parts to the finite collection of indicator <martingales> to obtain the time-dependent <Dynkin formula>: $g(s,X_s)-g(0,X_0)-\int_0^s(\partial_rg+Qg)(r,X_r)dr$ is a <martingale>. Therefore
$$
\mathbb E[f(X_T)\mid\mathcal F_s]=(P_{T-s}f)(X_s).
$$
This recovers the <Markov property> and the entire semigroup, proving the characterization. On countable spaces one uses nonexplosion and localization to justify the analogous unbounded-rate formulas.
For $d$-dimensional <Brownian motion>, the <Itô formula> shows that, for $f\in C_c^\infty(\mathbb R^d)$,
$$
f(B_t)-f(B_0)-\frac12\int_0^t\Delta f(B_s)\,ds=\int_0^t\nabla f(B_s)\cdot dB_s
$$
is a true <martingale>, since its bounded derivatives make the integral square-integrable on each bounded horizon. This is the <martingale problem for Brownian motion> with generator $L=\tfrac12\Delta$.
Here is why it also characterizes <Brownian motion>. Suppose a continuous adapted process $X$, with $X_0=x$, satisfies that <martingale problem>. Use smooth cutoff test functions equal to $x_i$ and $x_ix_j$ inside successively larger balls, and stop before leaving those balls. The coordinate tests show that $X^i-x_i$ are continuous <local martingales>; the product tests show that $X^iX^j-x_ix_j-\delta_{ij}t$ are local martingales. The <Itô product rule> consequently gives
$$
[X^i,X^j]_t=\delta_{ij}t:
$$
the difference between the two proposed compensators is a continuous finite-variation local martingale, hence constant and zero at time zero. For each $u\in\mathbb R^d$, another application of the <Itô formula> gives the exponential <local martingale>
$$
Z_t^u=\exp\!\left(iu\cdot(X_t-x)+\tfrac12|u|^2t\right).
$$
Its modulus is the deterministic quantity $e^{|u|^2t/2}$, so it is a true <martingale> on every bounded time interval. Taking its <conditional expectation> and dividing by its known value at time $s$ gives
$$
\mathbb E[e^{iu\cdot(X_t-X_s)}\mid\mathcal F_s]=e^{-|u|^2(t-s)/2}.
$$
The <conditional characteristic function> identifies a centered <multivariate normal distribution> of covariance $(t-s)I$, independent of the past. Together with continuity, this is the defining law of <Brownian motion>. This proves the <Lévy characterization of multidimensional Brownian motion> in the form needed for the generator characterization.
The analytic use of these <martingales> parallels the chain case. A <harmonic function> $h$ satisfies $\Delta h=0$, so it yields a stopped <local martingale> until the process leaves its domain of harmonicity. Bounded stopped observables permit <optional stopping> and solve boundary hitting problems. A solution of $\tfrac12\Delta v=-1$ with zero boundary values gives an expected exit time. For example, in the ball of radius $R$, $v(x)=(R^2-|x|^2)/d$ gives $\mathbb E_xT_R=(R^2-|x|^2)/d$; stopping $|B_t|^2-dt$ first at $T_R\wedge n$ proves finiteness and justifies the limiting calculation. The <Brownian local time> construction in Q4 is another example: the predictable compensators of approximations to $|B_t|$ recover its increasing occupation term.
We now prove the required three-dimensional <transience of Brownian motion in dimension at least three>, in the strong sense $|B_t|\to\infty$ <almost surely>. Start first at a point $x$ with $r<|x|<R$, and let $T$ be the first exit from this annulus. The <radial Laplacian> in dimension three gives
$$
\Delta\frac1{|x|}=\frac2{|x|^3}-\frac2{|x|^3}=0\qquad(x\ne0).
$$
Thus $|B_{t\wedge T}|^{-1}$ is a bounded <martingale>. The exit time is finite: stopping $|B_t|^2-3t$ gives $3\mathbb E(t\wedge T)\le R^2-|x|^2$. At exit, continuity puts $|B_T|$ at either $r$ or $R$. The <optional stopping theorem> therefore yields
$$
\frac1{|x|}=\frac1r\mathbb P_x(\tau_r<\tau_R)+\frac1R\mathbb P_x(\tau_R<\tau_r),
$$
and hence the <Brownian sphere-hitting probability in dimension three>
$$
\mathbb P_x(\tau_r<\tau_R)=\frac{|x|^{-1}-R^{-1}}{r^{-1}-R^{-1}}.
$$
As $R\to\infty$, the events $\{\tau_r<\tau_R\}$ increase to $\{\tau_r<\infty\}$: a continuous path is bounded on the finite interval preceding any finite inner hit. This argument assumes no transience. Consequently
$$
\boxed{\mathbb P_x(\tau_r<\infty)=r/|x|\quad(|x|>r).}
$$
To pass from a positive escape probability to almost-sure eventual escape, take <Brownian motion> started at zero. Let $\sigma_R$ be its first hit of the sphere of radius $R$. Stopping the squared-norm <martingale> proves $\mathbb E\sigma_R=R^2/3<\infty$. At $\sigma_R$, the <Strong Markov property> and the preceding hitting formula give
$$
\mathbb P(\exists t\ge\sigma_R:|B_t|\le r)=r/R\qquad(R>r).
$$
The <Strong Markov property> here follows from independent Brownian increments by approximating a stopping time from above with dyadic-valued stopping times and using path continuity. Choose $R_m=2^m r$. The exit times $\sigma_{R_m}$ increase to infinity, since a continuous path is bounded on every compact time interval. The events of a return to the closed ball of radius $r$ after $\sigma_{R_m}$ decrease and have <probabilities> $2^{-m}$. Their intersection has <probability> zero. Thus almost surely there is some finite exit time after which the path never again enters that ball. Apply this simultaneously to all positive integer $r$. This proves the <last visit to a bounded ball for three-dimensional Brownian motion> conclusion and therefore
$$
\boxed{|B_t|\longrightarrow\infty\quad\text{almost surely in dimension three}.}
$$
Back to article page
