Set $p=x^{k+1}-x^k={\alpha }^k{s}^k$ and $y=\nabla f(x^{k+1})-\nabla f(x^k)$. The Quasi-Newton secant equation is $B^{k+1}p=y$. If $B^{k+1}$ is a positive-definite symmetric matrix and $p\ne 0$, it follows thatThe weak curvature part of the Wolfe conditions ensures precisely this necessary condition:The last product is positive because ${\alpha }^k>0$, ${\beta }_2-1<0$, and $s^k$ is a descent direction. This explains why a Quasi-Newton method often uses the Wolfe conditions rather than only the Armijo condition.

If $g=\nabla f(x)\ne 0$, the positive-definite symmetric matrix $B$ is an invertible matrix and $s=-B^{-1}g\ne 0$. Multiplying $Bs=-g$ by $s^T$ givesHence $s$ is a descent direction. If $g=0$, then $s=0$ and the algorithm has already reached a stationary point; the strict descent assertion applies before termination.

First suppose $0<r<\rho =|z_0|$. For every $R>\rho$, the planar Brownian annulus hitting probability givesHence ${\tau }_r<\infty$ almost surely. This uses increasing outer radii; letting the inner radius tend to zero instead proves the different claim in part (ii).

For $r>\rho$, the disk-exit estimate in (i), with outer radius $r$, directly yields $\mathbb{E}{\tau }_r\le (r^2-{\rho }^2)/2<\infty$.

For $r=\rho$, retain the definition ${\tau }_r=\inf \{t>0:|B_t|=r\}$. The first case shows that radius $\rho /2$ is reached in finite time. Starting there, the Strong Markov property of Brownian motion and the disk-exit argument show that radius $\rho$ is subsequently reached in finite time. Thus the infimum over positive hitting times is finite in this case as well. Altogether,The conclusion can also be made simultaneous in all $r>0$: intersect the probability-one events for positive rational radii, then use continuity to cross each intermediate radius.

Fix an outer radius $R>\rho =|z_0|$. By continuity, reaching zero before radius $R$ requires first reaching every radius $\epsilon \in (0,\rho )$. The planar Brownian annulus hitting probability from (i) therefore impliesThus $\mathbb{P}({\tau }_0<{\tau }_R)=0$ for every fixed $R>\rho$.

On any continuous sample path that reaches zero in finite time, the radius has a finite maximum before that time. There is therefore some integer $m>\rho$ for which zero is reached before radius $m$. ConsequentlyEvery event on the right has probability zero, soThis proves that planar Brownian motion avoids a fixed point even though it can reach arbitrarily small positive radii.

Let $\rho =|z_0|$ and $f(x,y)=\log \sqrt{x^2+y^2}$. On the annulus $r<\sqrt{x^2+y^2}<R$, its gradient and Laplace operator areThus $f$ is a harmonic function there. The stopped Itô formula givesThe sum of the squared coefficients is $|B_s{|}^{-2}\le r^{-2}$ before $\tau$. The Itô isometry therefore makes the stochastic term a true square-integrable martingale on every finite interval. Continuity prevents overshoot at exit, so $\log r\le \log |B_{t\wedge \tau }|\le \log R$. This proves the required bounded martingale property.

Before taking the terminal expectation, prove that $\tau$ is finite. Set ${\theta }_R=\inf \{t\ge 0:|B_t|=R\}$. The Itô formula for $|B{|}^2$ shows thatis a martingale, since its stopped stochastic integrands are bounded. Taking expectations yieldsThe monotone convergence theorem gives $\mathbb{E}{\theta }_R\le (R^2-{\rho }^2)/2<\infty$. Hence $\tau \le {\theta }_R<\infty$ almost surely.

The bounded convergence theorem, equivalently the optional stopping theorem applied to this bounded martingale, gives $\mathbb{E}\log |B_{\tau }|=\log \rho$. The two exit events partition the sample space up to a null set. Writing $p=\mathbb{P}({\tau }_r<{\tau }_R)$,Solving gives the planar Brownian annulus hitting probability

Let $X=(X^1,\dots ,X^d)$ be a continuous semimartingale, with $X^j=X_0^j+M^j+A^j$, where $M^j$ is a zero-starting continuous local martingale and $A^j$ is a continuous finite-variation process starting at zero. For $f\in C^{1,2}([0,\infty )\times {\mathbb{R}}^d)$, the Itô formula isHere $[M^j,M^k]$ is the quadratic covariation; the finite-variation process parts contribute no such term. The integrals against the finite-variation processes are understood pathwise, and those against the continuous local martingales are Itô integrals after localization.

For $d$-dimensional Brownian motion with independent standard coordinates, $d[B^j,B^k{]}_s={\delta }_{jk}ds$. In particular, if $f\in C^2$ is independent of time,where $\Delta$ is the Laplace operator. The same formulas hold up to exit from an open domain on which $f$ is smooth.

The optional stopping theorem for a right-continuous martingale $M$ on a filtered probability space satisfying the usual conditions for a filtration states that, for bounded stopping times $\sigma \le \tau$,For a submartingale the conditional equality becomes $\mathbb{E}[M_{\tau }\mid {\mathcal{F}}_{\sigma }]\ge M_{\sigma }$, and for a supermartingale it becomes the reverse inequality.

For a uniformly integrable martingale, boundedness of the stopping times can be removed, with the terminal limit used if a stopping time is infinite. A useful application is to a bounded stopped martingale $M_{t\wedge \tau }$ with $\tau <\infty$ almost surely: apply the bounded-time theorem first and then the bounded convergence theorem to obtain $\mathbb{E}{M}_{\tau }=\mathbb{E}{M}_0$. Finiteness of $\tau$ alone is not a sufficient hypothesis for an arbitrary martingale.

Part (ii) represents the process as an Iterated Itô integral:For a centered normal random variable $B_s$ of variance $s$,Consequently, for every finite $T$,The Itô integral of this square-integrable stochastic integrand is a true square-integrable martingale on $[0,T]$. Since $T$ is arbitrary,The last identity is the Itô isometry.

Interpret the ordered Iterated Itô integral recursively as $I_0(t)=1$ andThe diagonal boundary of the ordered region does not affect this recursive definition. For $n=1$, $I_1(t)=B_t$. For $n=2$, part (a)(iii) gives $I_2(t)=\frac{1}{2}(B_t^2-t)$. For $n=3$, use parts (b)(ii) and (b)(iii):Thus the three Probabilists' Hermite polynomials giveThe right sides are interpreted as their polynomial extensions at $t=0$. These space-time Hermite polynomials arise here directly from ordered Iterated Itô integrals.

Put $M_t=B_t^3-3t{B}_t$. It is an adapted process and is integrable at every finite time, because normal random variables have finite absolute moments of every order. Fix $0\le s<t$ and let $D=B_t-B_s$. Then $D$ is independent of ${\mathcal{F}}_s$, has mean zero, variance $t-s$, and third moment zero. Expanding the cube inside the conditional expectation givesAlso $\mathbb{E}[B_t\mid {\mathcal{F}}_s]=B_s$. ThereforeThese are exactly the defining conditions for a martingale.

Summing the exact cubic increment identity givesThe first sum tends to zero in L2 space. Indeed, odd moments of centered normal random variables vanish, a centered normal random variable of variance $v$ has sixth moment $15v^3$, and independence of the Brownian increments givesThe second sum converges in L2 space to $3{\int }_0^t{B}_{s}ds$ by (i).

For the last sum, first note that $\mathbb{E}{\int }_0^t{B}_s^{4}ds={\int }_0^{t}3s^{2}ds=t^3$, so $B_s^2$ is a square-integrable stochastic integrand. If $s=t_j+u$ and $D=B_s-B_j$, then $D$ is independent of $B_j$, with $D\sim N(0,u)$. HenceIt follows thatThe Itô isometry therefore makes ${\sum }_j{B}_j^2\Delta B_j$ converge in L2 space to ${\int }_0^t{B}_s^{2}d{B}_s$, with bounded truncations supplying the initial simple predictable processes if needed. Taking limits in the cubic identity provesThe proof uses only increment algebra, moments of normal random variables and the definition of the Itô integral.

Apply the exact discrete product identityand sum over the deterministic partition of an interval. Since $B_0=0$,The left side converges in L2 space to ${\int }_0^{t}sd{B}_s$ by the definition and Itô isometry, sinceThe right-endpoint Riemann sum converges in L2 space to ${\int }_0^t{B}_{s}ds$: its squared error is at mostConsequently,As in (a)(iii), continuity extends fixed-time equality to equality of the continuous processes.

To establish the Brownian weighted quadratic variation in L2 for the deterministic partition of an interval, decomposewhere $U_{\pi }={\sum }_j{B}_j((\Delta B_j{)}^2-\Delta t_j)$ and $V_{\pi }={\sum }_j{B}_j\Delta t_j-{\int }_0^t{B}_{s}ds$.

The summands in $U_{\pi }$ are martingale differences: $B_j$ is ${\mathcal{F}}_{t_j}$-measurable, and the following Brownian increment is independent of ${\mathcal{F}}_{t_j}$. Their conditional means are zero, so distinct summands are orthogonal in L2 space. Since $\mathbb{E}{B}_j^2=t_j$ and $\mathbb{E}[((\Delta B_j{)}^2-\Delta t_j{)}^2]=2(\Delta t_j{)}^2$,For the Riemann sum error, the Cauchy-Schwarz inequality givesThus the triangle inequality in L2 space provesThe time integration variable is $s$.

Fix $t>0$, a deterministic partition of an interval $\pi$, and write $\eta (s)=t_j$ on $[t_j,t_{j+1})$. Its mesh is $|\pi |={\max }_j(t_{j+1}-t_j)$. The Brownian increment variance givesThus the left-endpoint sums ${\sum }_j{B}_{t_j}\Delta B_j$ converge in L2 space to the Itô integral. Although these step coefficients are unbounded, clipping them at finite levels first makes them bounded simple predictable processes; the Itô isometry and their finite second moments remove the clipping.

The elementary identity $B_{t_{j+1}}^2-B_{t_j}^2=2B_{t_j}\Delta B_j+(\Delta B_j{)}^2$ givesThe squared Brownian increments are independent, with means $\Delta t_j$ and variances $2(\Delta t_j{)}^2$, since a centered normal random variable of variance $v$ has fourth moment $3v^2$. ThereforePassing to the convergence in L2 limits yieldsThis holds for each fixed $t$; continuous sample paths of both sides extend equality from rational times to all $t$ on one event of probability one.

For a bounded simple predictable process $H$ and a continuous martingale $M\in {\mathcal{H}}^{2,c}$, the Itô isometry statesHere $⟨M⟩$ is the predictable quadratic variation, equal to $[M]$ for a continuous martingale. On the whole half-line, the terminal form isThe Itô integral starts at zero even when $M_0\ne 0$. For Brownian motion, $d⟨B{⟩}_s=ds$.

For completeness, cross terms of the finite increment sum have expectation zero by conditioning on the earlier endpoint of the later increment. Also $M^2-⟨M⟩$ is a martingale, so the remaining terms satisfyThis proves the displayed identity without a change-of-variables formula. Completing bounded simple predictable processes in the norm $(\mathbb{E}\int H^{2}d⟨M⟩{)}^{1/2}$ defines the Itô integral of every square-integrable stochastic integrand. This completion will justify the unbounded Brownian integrands below.

Let $0=t_0<t_1<\cdots <t_m<\infty$ and let ${\xi }_j$ be bounded ${\mathcal{F}}_{t_j}$-measurable random variables. For the bounded simple predictable processthe Itô integral against a continuous martingale $M\in {\mathcal{H}}^{2,c}$ is defined by the increment sumThe same definition applies to elementary predictable processes with stopping-time intervals, replacing $t_j$ by ordered stopping times $T_j$ and taking bounded ${\mathcal{F}}_{T_j}$-measurable coefficients. Refinement of the intervals leaves the increment sum unchanged. The initial value of the integral is zero, and its martingale property follows from the conditional expectation identity for subsequent increments of $M$.

Write $e_t=1_{[0,t]}$ as an element of L2 space. Parts (iii) and (iv) show that $B_t=W{e}_t$ is a centered Gaussian process with covariance functionIn particular $B_t-B_s\sim N(0,|t-s|)$. The fourth moment of a centered normal random variable of variance $v$ is $3v^2$, soThe Kolmogorov continuity theorem, with moment exponent $4$ and time exponent $2>1$, provides a continuous modification $\hat{B}$. At each finite set of times this modification of a stochastic process has the same joint law as $B$.

For $0\le t_0<t_1<\cdots <t_m\le 1$, the successive increments are Jointly Gaussian. The covariance of two increments on disjoint intervals equals the inner product of their disjoint interval indicators and is zero. Zero cross-covariances of Jointly Gaussian variables imply independence, and each increment has variance $t_j-t_{j-1}$. Finally $e_0=0$ in L2 space, so ${\hat{B}}_0=0$ almost surely. ThereforeIndependence from the whole past in its natural Brownian filtration follows by extending finite-dimensional independence to the generated sigma-algebra.

Use the same finite truncation $T_m(h)={\sum }_{j\le m}⟨h,e_j⟩A_j$ for $f$ and $g$. Since $\mathbb{E}[A_i{A}_j]={\delta }_{ij}$ by independence, zero means and unit variances,Here ${\delta }_{ij}$ is the Kronecker delta. The Cauchy-Schwarz inequality and convergence in L2 justify passing to the limit in the product's expectation. The Parseval identity for a Hilbertian basis now givesBecause each $Wh$ has mean zero, this is also its covariance identity. Thus $W$ is an isonormal Gaussian process over $\mathcal{H}$.

For any $h_1,\dots ,h_m\in \mathcal{H}$ and real $u_1,\dots ,u_m$, linearity of the finite partial sums and convergence in L2 giveBy part (ii), this random variable has the normal distributionA finite family is Jointly Gaussian precisely when every real linear combination has a possibly degenerate normal distribution. Thus every finite subfamily of $(Wh{)}_{h\in \mathcal{H}}$ is Jointly Gaussian, which is the meaning of joint Gaussianity for this collection.

Enumerate the orthonormal basis consisting of $H_0$ and the Haar wavelets $H_{n,k}$ as $(e_j{)}_{j\ge 1}$, and enumerate the corresponding independent standard normal random variables as $(A_j{)}_{j\ge 1}$. Set $c_j=⟨h,e_j{⟩}_{\mathcal{H}}$. The Parseval identity for a Hilbertian basis givesThe partial sums $T_m={\sum }_{j=1}^m{c}_j{A}_j$ form a martingale for $\sigma (A_1,\dots ,A_m)$, since the next summand is independent of the past and has mean zero. Moreover,The martingale convergence theorem for a martingale bounded in L2 space proves convergence almost surely and convergence in L2. Equivalently one may apply the Kolmogorov convergence theorem to the summable variances $c_j^2$.

A finite linear combination of independent normal random variables is normal. Its characteristic function is $\mathbb{E}{e}^{iu{T}_m}=\exp (-u^2{\sum }_{j\le m}{c}_j^2/2)$. Passing to the limit identifiesFor $h=0$ this is the constant zero random variable. Each fixed $h$ has its own permissible null set; simultaneous pathwise convergence for all $h\in \mathcal{H}$ is unnecessary.