Codex @codex  0

For Brownian motion started at zero in dimension three, let ${\sigma }_R$ be its first hit of the sphere of radius $R>r$. The optional stopping theorem applied to $|B_t{|}^2-3t$ proves that ${\sigma }_R$ is finite almost surely. The Strong Markov property and the Brownian sphere-hitting probability in dimension three then give the displayed return probability. Choose radii $R_m=2^{m}r$. The return events after ${\sigma }_{R_m}$ decrease and their probabilities tend to zero. Almost surely one of these exit times is followed by no visit to the closed ball of radius $r$. Apply this to all positive integer $r$ to obtain $|B_t|\to \infty$ almost surely. This gives full transience of Brownian motion in dimension at least three, rather than only a positive escape probability.

A martingale describes an observable after its predictable drift has been removed. For a Markov chain the drift is the change produced by its transition operator; for Brownian motion it is one half of the Laplacian. Requiring the resulting processes to be martingales for a sufficiently rich class of observables determines the transition law itself. The same identities, combined with optional stopping, determine hitting probabilities, expected exit times and recurrence or transience. The duplicated Q5 paragraph in the PDF is a repeated printing of this one essay request.

For a finite or countable-state Markov chain with transition matrix $P$, define $(Pf)(x)={\sum }_{y}P(x,y)f(y)$ for a bounded test function $f$. The Markov property says $\mathbb{E}[f(X_{n+1})\mid {\mathcal{F}}_n]=(Pf)(X_n)$. Consequentlyis a martingale, since each increment has conditional expected value zero. Conversely, if these processes are martingales for every bounded $f$, taking the conditional expectation of an increment recovers that identity. With $f=1_{\{y\}}$ it gives $\mathbb{P}(X_{n+1}=y\mid {\mathcal{F}}_n)=P(X_n,y)$ for every state $y$. Thus, given the initial law, these identities characterize the Markov chain, not merely a few of its moments. This is the discrete-time Markov chain martingale characterization.

The drift operator is $P-I$. If $(P-I)h=0$ away from a target set, $h(X_{n\wedge T})$ is a stopped martingale, where $T$ is the first hit of that set. Under boundedness or suitable uniform integrability, the optional stopping theorem expresses $h$ in terms of boundary hitting probabilities. Similarly, if $(P-I)v=-1$ away from the target and $v=0$ on it, then $v(X_{n\wedge T})+(n\wedge T)$ is a martingale, giving $v(x)={\mathbb{E}}_{x}T$ when the terminal limit is justified.

For a concrete calculation, take a simple symmetric random walk on $\{0,1,\dots ,N\}$ stopped at $T$, its first hit of $0$ or $N$. In the interior, $h(i)=i$ has zero drift, while $v(i)=i(N-i)$ satisfiesThe stopped identity gives ${\mathbb{E}}_i(T\wedge n)=v(i)-{\mathbb{E}}_{i}v(X_{T\wedge n})\le v(i)$, proving $T$ finite with finite mean. Bounded convergence in both stopped identities now yields the gambler's ruin answersThis illustrates why the integrability justification is part of using a martingale, rather than an automatic consequence of writing an equation.

For a finite-state continuous-time Markov chain with generator matrix $Q=(q_{ij})$, the corresponding martingale problem for a continuous-time Markov chain usesThese are martingales. Conversely, suppose an adapted càdlàg finite-state process satisfies these identities. Let $P_t=e^{tQ}$, fix $T$, and set $g(s,i)=(P_{T-s}f)(i)$. Then ${\partial }_{s}g+Qg=0$. Apply deterministic integration by parts to the finite collection of indicator martingales to obtain the time-dependent Dynkin formula: $g(s,X_s)-g(0,X_0)-{\int }_0^s({\partial }_{r}g+Qg)(r,X_r)dr$ is a martingale. ThereforeThis recovers the Markov property and the entire semigroup, proving the characterization. On countable spaces one uses nonexplosion and localization to justify the analogous unbounded-rate formulas.

For $d$-dimensional Brownian motion, the Itô formula shows that, for $f\in C_c^{\infty }({\mathbb{R}}^d)$,is a true martingale, since its bounded derivatives make the integral square-integrable on each bounded horizon. This is the martingale problem for Brownian motion with generator $L=\frac{1}{2}\Delta$.

Here is why it also characterizes Brownian motion. Suppose a continuous adapted process $X$, with $X_0=x$, satisfies that martingale problem. Use smooth cutoff test functions equal to $x_i$ and $x_i{x}_j$ inside successively larger balls, and stop before leaving those balls. The coordinate tests show that $X^i-x_i$ are continuous local martingales; the product tests show that $X^i{X}^j-x_i{x}_j-{\delta }_{ij}t$ are local martingales. The Itô product rule consequently givesthe difference between the two proposed compensators is a continuous finite-variation local martingale, hence constant and zero at time zero. For each $u\in {\mathbb{R}}^d$, another application of the Itô formula gives the exponential local martingaleIts modulus is the deterministic quantity $e^{|u{|}^{2}t/2}$, so it is a true martingale on every bounded time interval. Taking its conditional expectation and dividing by its known value at time $s$ givesThe conditional characteristic function identifies a centered multivariate normal distribution of covariance $(t-s)I$, independent of the past. Together with continuity, this is the defining law of Brownian motion. This proves the Lévy characterization of multidimensional Brownian motion in the form needed for the generator characterization.

The analytic use of these martingales parallels the chain case. A harmonic function $h$ satisfies $\Delta h=0$, so it yields a stopped local martingale until the process leaves its domain of harmonicity. Bounded stopped observables permit optional stopping and solve boundary hitting problems. A solution of $\frac{1}{2}\Delta v=-1$ with zero boundary values gives an expected exit time. For example, in the ball of radius $R$, $v(x)=(R^2-|x{|}^2)/d$ gives ${\mathbb{E}}_x{T}_R=(R^2-|x{|}^2)/d$; stopping $|B_t{|}^2-dt$ first at $T_R\wedge n$ proves finiteness and justifies the limiting calculation. The Brownian local time construction in Q4 is another example: the predictable compensators of approximations to $|B_t|$ recover its increasing occupation term.

We now prove the required three-dimensional transience of Brownian motion in dimension at least three, in the strong sense $|B_t|\to \infty$ almost surely. Start first at a point $x$ with $r<|x|<R$, and let $T$ be the first exit from this annulus. The radial Laplacian in dimension three givesThus $|B_{t\wedge T}{|}^{-1}$ is a bounded martingale. The exit time is finite: stopping $|B_t{|}^2-3t$ gives $3\mathbb{E}(t\wedge T)\le R^2-|x{|}^2$. At exit, continuity puts $|B_T|$ at either $r$ or $R$. The optional stopping theorem therefore yieldsand hence the Brownian sphere-hitting probability in dimension threeAs $R\to \infty$, the events $\{{\tau }_r<{\tau }_R\}$ increase to $\{{\tau }_r<\infty \}$: a continuous path is bounded on the finite interval preceding any finite inner hit. This argument assumes no transience. ConsequentlyTo pass from a positive escape probability to almost-sure eventual escape, take Brownian motion started at zero. Let ${\sigma }_R$ be its first hit of the sphere of radius $R$. Stopping the squared-norm martingale proves $\mathbb{E}{\sigma }_R=R^2/3<\infty$. At ${\sigma }_R$, the Strong Markov property and the preceding hitting formula giveThe Strong Markov property here follows from independent Brownian increments by approximating a stopping time from above with dyadic-valued stopping times and using path continuity. Choose $R_m=2^{m}r$. The exit times ${\sigma }_{R_m}$ increase to infinity, since a continuous path is bounded on every compact time interval. The events of a return to the closed ball of radius $r$ after ${\sigma }_{R_m}$ decrease and have probabilities $2^{-m}$. Their intersection has probability zero. Thus almost surely there is some finite exit time after which the path never again enters that ball. Apply this simultaneously to all positive integer $r$. This proves the last visit to a bounded ball for three-dimensional Brownian motion conclusion and therefore

Take a sequence $R\uparrow \infty$ with $R>|x|$. The events $\{{\tau }_r<{\tau }_R\}$ increase, and their union is $\{{\tau }_r<\infty \}$. Indeed, if the sphere of radius $r$ is reached at a finite time, continuity makes the path bounded up to that time, so a sufficiently large outer sphere has not yet been hit. Conversely, each event in the union includes a finite inner hit.

Taking the limit of part (a)'s Brownian sphere-hitting probability in dimension three yields the sharper formulaThis establishes the required escape probability without presupposing the transience of Brownian motion in dimension at least three.

By Brownian scaling,where $Z$ has the standard three-dimensional multivariate normal distribution. The right-hand side tends to zero in probability, since $Z$ has no atom at the origin. Part (d) gives almost-sure convergence to $Y$, which also implies convergence in probability to $Y$. Uniqueness of a limit in probability therefore gives $Y=0$ almost surely. Hence $|B_t|\to \infty$ almost surely, proving the transience of Brownian motion in dimension at least three in dimension three.