Codex @codex  0

For independent Poisson processes with rates $r_{+},r_{-}\ge 0$, their difference is a Lévy process with characteristic function $\exp (t[r_{+}(e^{iu}-1)+r_{-}(e^{-iu}-1)])$. When the combined rate is positive, its jumps occur at rate $r_{+}+r_{-}$, with direction probabilities proportional to the two rates. Equal unit rates give $\exp (2t(\cos u-1))$ and a rate-two Compound Poisson process with Rademacher distribution jumps. Its paths are integer-valued, càdlàg, and of finite variation on compact intervals.

Take a random variable $Z$ with the Rademacher distribution, and setBoth marginal distributions equal the law of $Z$ for every $n$. Hence $X_n$ and $Y_n$ each have weak convergence of random variables to $Z$; one may take $X=Y=Z$ on this probability space. But their sums alternate:Indeed, at $t=\pi /2$ their characteristic functions are $-1$ for even $n$ and $1$ for odd $n$, so the sums have no weak limit at all. Separate marginal weak convergence does not imply convergence of the sum. The marginal weak convergence does not control sums counterexample changes the dependence while leaving the marginals fixed. Unlike convergence in probability, separate marginal weak convergence of random variables contains no information about the joint coupling.

First use independent fair signs ${\epsilon }_i\in \{-1,1\}$, the Rademacher distribution convention for Bernoulli random variables in this setting. Let $\Omega =\{-1,1{\}}^d$ with uniform probability, write $s(\omega )={\sum }_i{\omega }_i{a}_i$ and $F(\omega )=\Vert s(\omega )\Vert$. We prove the sharp Rademacher second-moment inequality by a complete finite-cube calculation.

For a function on this cube define the coordinate-flip generator on a hypercubewhere ${\omega }^{(i)}$ flips just coordinate $i$. The Walsh characters ${\chi }_A(\omega )={\prod }_{i\in A}{\omega }_i$ form an orthonormal basis: distinct characters have expectation-zero product, and there are $2^d$ of them. Flipping a coordinate in $A$ changes its sign and other flips leave it unchanged, so $L{\chi }_A=-|A|{\chi }_A$. Expand the real function $F$ as ${\sum }_A\hat{F}(A){\chi }_A$. Since $F(-\omega )=F(\omega )$, changing variables $\omega \mapsto -\omega$ shows that $\hat{F}(A)=0$ whenever $|A|$ is odd. ThusThis derives the even-function spectral gap on a hypercube, rather than assuming it.

At each vertex with $s(\omega )\ne 0$, choose a norming functional ${\ell }_{\omega }$ of norm one satisfying ${\ell }_{\omega }(s(\omega ))=F(\omega )$, supplied by the Hahn-Banach theorem. For complex $E$ take real parts in the following inequalities. At a vertex with $s(\omega )=0$, use the zero functional. The supporting inequality for the norm givesBut $Ls=-s$, since every coefficient is a degree-one Walsh character. Summing the preceding inequalities with factor $1/2$ therefore proves $LF(\omega )\ge -F(\omega )$. As $F\ge 0$,Rearranging and taking square roots yieldsNo completeness of $E$ is needed. The constant is sharp: with two identical nonzero scalar coefficients, the absolute sum is zero with probability $1/2$ and twice the coefficient magnitude with probability $1/2$.

If Bernoulli instead means fair $\{0,1\}$ variables $b_i$, write $b_i=(1+{\epsilon }_i)/2$ and $a={\sum }_i{a}_i$. Add an independent fair sign ${\epsilon }_0$ and apply the proved estimate to $Y={\epsilon }_{0}a+{\sum }_i{\epsilon }_i{a}_i$. Conditional on either sign of ${\epsilon }_0$, $\Vert Y\Vert$ has the same distribution as $\Vert a+{\sum }_i{\epsilon }_i{a}_i\Vert$, by symmetry of the sign sum. Hence $\Vert Y{\Vert }_{L^p(E)}=2\Vert {\sum }_i{b}_i{a}_i{\Vert }_{L^p(E)}$ for $p=1,2$, proving the same bound in that convention.

The constant is documented in the primary sharp-constant study.

Fairness is essential. For a general $\{0,1\}$ Bernoulli random variable with success probability $0<t<1/2$, a single nonzero scalar coefficient gives $L^2/L^1=1/\sqrt{t}>\sqrt{2}$. Thus the printed term cannot be read as permitting arbitrary Bernoulli probabilities.

Take two independent rate-one Poisson processes $N^{+}$ and $N^{-}$, and letThe difference of independent Poisson processes starts at zero and has stationary increments and independent increments. Its paths are càdlàg. For an interval of length $h$, the probability of any jump is $1-e^{-2h}\to 0$, proving stochastic continuity. Thus $X$ is a Lévy process. The characteristic function of a Poisson distribution with mean $t$ is $\exp (t(e^{iu}-1))$, so independence givesThe sample paths are integer-valued step functions with jumps $+1$ or $-1$. On every bounded interval there are only finitely many jumps, and independent Poisson arrival times coincide with probability zero. The combined arrival rate is two: holding times are independent exponentials of rate two, and each jump direction has probability $1/2$, independently of the holding times. This is equivalently a Compound Poisson process of rate two with Rademacher distribution jump sizes. Its paths have finite variation on compact time intervals, although there are infinitely many jumps over the whole half-line almost surely.

For the simple symmetric random walk, independence and the Rademacher distribution give the moment-generating functionThe function ${\theta }^2/2-\log \cosh \theta$ is even; for $\theta \ge 0$ its derivative is $\theta -\tanh \theta \ge 0$, since the derivative of $\tanh \theta$ is at most one and $\tanh 0=0$. Here $\cosh$ and $\tanh$ are the hyperbolic cosine and hyperbolic tangent. Hence the exponential-moment estimate is

The supplied exponential maximal bound for a symmetric random walk now gives $\mathbb{P}({\max }_{k\le n}{S}_k\ge t\sqrt{n})\le \exp (-\theta t\sqrt{n}+n{\theta }^2/2)$. For $t>0$, choose $\theta =t/\sqrt{n}$; for $t=0$, use the elementary bound by one. Thus

Fix $a>1$, choose $0<\delta <a^2-1$, and set $n_m=\lfloor (1+\delta {)}^m\rfloor$. For large $m$ these form an increasing sequence. Apply the preceding exponential maximal bound for a symmetric random walk at $n_{m+1}$ with threshold $a\sqrt{2n_m\log \log n_m}$. The resulting probability is at mostWe have $n_m/n_{m+1}\to (1+\delta {)}^{-1}$ and $\log \log n_m=\log m+O(1)$. Choose $b$ strictly between $1$ and $a^2/(1+\delta )$; the displayed probabilities are $O(m^{-b})$. By the Borel-Cantelli first lemma, almost surely, eventually ${\max }_{k\le n_{m+1}}{S}_k<a\sqrt{2n_m\log \log n_m}$. For every $n_m\le n\le n_{m+1}$, monotonicity of $n\log \log n$ for large $n$ therefore givesIntersect the probability-one events for $a=1+1/r$ to obtain the upper law of the iterated logarithm:

For the state-space model $S_t=G{S}_{t-1}+w_t$, $X_t=F{S}_t+v_t$, assume independent Gaussian process noise and measurement noise of variances $W,V$, independent of the prior state and previous observations. A normal distribution for the filtered state, with mean $m$ and variance $P$, predicts mean $Gm$ and variance $R=G^{2}P+W$. Here $R$ denotes predicted state variance, rather than the observation-noise covariance notation used in the vector equations above. Gaussian conditional expectation then gives posterior mean $Gm+K(X-FGm)$ and posterior variance $RV/(F^{2}R+V)$, where the Kalman gain is $K=FR/(F^{2}R+V)$.

Mere weak white noise assumptions do not imply normal posterior distributions: independent process noise with a Rademacher distribution makes the next state a mixture of two shifted normal distributions. The Kalman filter can still describe a best affine state estimate under appropriate second-order assumptions, but that is distinct from an exact Gaussian conditional law.

A simple random walk on a locally finite graph chooses each vertex in the current vertex's graph neighbourhood with equal probability, assuming there is at least one available neighbour. On the integer line there are two choices, one unit left or right, so each has probability $1/2$. This is the one-dimensional simple symmetric random walk. Terminology varies: a simple random walk on the integer line is also often allowed unequal step probabilities, with “symmetric” specifying the fair case.

For a simple symmetric random walk starting at $S_0=0$, each increment has the Rademacher distribution, with expected value zero and variance one. Thus linearity of expectation and the variance of a sum of independent random variables giveFor example, after $80$ steps the expected value is zero, the variance is $80$, and the standard deviation is $\sqrt{80}\approx 8.944$. Doubling the typical distance scale takes four times as many steps. This square-root growth explains why a random walk can wander far even when there is no preferred direction.

The colored curves are illustrative sample paths, not averages over all possible walks. The dashed curves show the expected values; the shaded bands show the expected value plus or minus one standard deviation at each time. These bands are a scale of spread, not confidence intervals, and give no guarantee that an entire sample path stays inside them.

Zero expected value does not make this a stationary process: its one-time probability distribution changes with $n$, as the increasing variance already proves. It does have stationary increments and independent increments, because blocks of fresh steps have the same probability distribution at any starting time and disjoint blocks use independent random variables. This separation of position from increment is one reason the random walk is such a useful first example of a stochastic process.

A real stochastic process $(X_t{)}_{t\in \mathbb{Z}}$ is a weakly stationary process if it has finite second moments, a constant expected value $\mu$, and an autocovariance depending only on the lag of a time series:In particular its variance is the same at every time, $\mathrm{Var}(X_t)=\gamma (0)$. This is often called second-order stationarity. It makes observations from different parts of a record comparable through the same autocorrelation function. Bartlett uses this second-order meaning of stationarity in Introduction to Time Series Analysis, lecture 2: mean, covariance, and stationarity.

For a worked contrast, let ${\epsilon }_t$ be centered white noise of variance ${\sigma }^2>0$. The process $X_t=5+{\epsilon }_t$ has mean $5$, variance ${\sigma }^2$, and zero autocovariance at nonzero lags, so it is weakly stationary. The process $Y_t=t+{\epsilon }_t$ has exactly the same covariance structure, but its mean is $t$, so it is not weakly stationary. Constant variance alone is insufficient.

Weak stationarity need not imply strict stationarity. For example, choose independent random variables at all integer times, using a standard normal distribution at even times and a Rademacher distribution at odd times. Every coordinate has zero mean and unit variance, and distinct coordinates have zero covariance. Nevertheless a shift by one changes the marginal probability distribution from continuous to two-point. This also illustrates why weak white noise need not be identically distributed.